Event notification in interconnected content-addressable storage systems

ABSTRACT

Some of the embodiments herein provide a seamless cloud of storage. This storage may be content-addressable storage. An end application may or may not be exposed to the fact that content-addressable storage is used. Various embodiments herein provide event notification, which may allow applications or users to subscribe to particular events (such as storage of an X-ray by a particular entity). Some embodiments provide for a shared archive. A shared archive may provide homogeneous access to medical data, etc. that was previously stored into the CAS cloud by heterogeneous applications, varied data types, etc. Additionally, embodiments herein allow for the creation and distribution of virtual packages. For example, a user may create a virtual package for all images related to a patient so that she may have a virtual package of all of her medical data to present to a referring physician.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/586,597, titled Event Notification In InterconnectedContent-Addressable Storage Systems, filed Dec. 30, 2014, which is acontinuation of U.S. patent application Ser. No. 13/092,229, titledEvent Notification In Interconnected Content-Addressable StorageSystems, filed Apr. 22, 2011, issued as U.S. Pat. No. 8,930,470 on Jan.6, 2015, which claims the benefit of U.S. Provisional Application No.61/327,556, titled Management of Interconnected Content-AddressableStorage Systems, filed Apr. 23, 2010. Reference is also made to: U.S.patent application Ser. No. 13/092,243, titled “Shared Archives inInterconnected Content-Addressable Storage Systems,” filed Apr. 22,2011, now U.S. Pat. No. 8,799,221; and U.S. patent application Ser. No.13/092,253, titled “Management of Virtual Packages of Medical Data inInterconnected Content-Addressable Storage Systems,” also filed Apr. 22,2011, now U.S. Pat. No. 8,407,244. The entire disclosure of each itemlisted above is incorporated by reference herein and made part of thisdisclosure, for all purposes, for all that each contains.

FIELD

This disclosure relates to content-addressable storage (CAS) forhandling, storing, and distributing medical imaging information and,more specifically, to event notification in interconnectedcontent-addressable storage systems.

BACKGROUND

Many files stored in computer systems contain data that is not expectedto change over time. In some systems, the percentage of files that areexpected to remain unchanged can range up to 90% of all of the storedfiles. Examples of data and files that are expected to remain unchangedinclude medical images, images of cancelled bank checks, imagescollected by oil and gas exploration, surveillance videos, filescontaining television news clips, and many types of archival andhistorical data. Other files are expected to change regularly, such as adatabase file, a word processing document that is being edited, and anytype of file that represents current state, such as a file holdingcumulative email messages as they arrive.

Stored files must be accessible. Further, they must be accessiblewhether it's the kind of data that changes quickly, such as a filestoring current email, or whether it is the kind of data that will notchange much over time, such as medical images. CAS technology can beused to store different types of data including, by way of example, datathat does not change over time. Generally, a “handle” (not necessarilythe location of the file in a directory) or a GUID (globally uniqueidentifier) is created for each stored object. This handle can becreated based on known techniques, such as hashing.

Current storage systems have a number of issues. The options forproduction of groupings of files may be limited. Another issue withcurrent systems is that the heterogeneous storage of data (e.g., but anumber of different medical imagers and reporting systems), can makeaccessing stored data difficult—both because of the distributed natureof the storage and because of the heterogeneous nature of the storage ofthe data.

These and other issues are addressed by the embodiments describedherein. Some embodiments address one or more of these issues, whileothers address a different sub set of issues.

SUMMARY

Embodiments of the systems, methods, and devices described hereinovercome problems of the prior art and enable management ofinterconnected content-addressable storage systems.

In some embodiments, event notification in an interconnectedcontent-addressable storage system is provided. Embodiments may includereceiving a request for event notification of a particular type of eventat a first content-addressable storage server (implemented on one ormore computing devices) from a first application. Receiving a request toperform an action associated with an event of the particular event typeat a second content-addressable storage server, where the first andsecond content-addressable storage servers are distinct and where thefirst and second content-addressable storage servers are both part of acontent-addressable storage cloud. The embodiments may also includeperforming the action associated with the event of the particular eventtype at the second content-addressable storage server, and, uponperformance of the action associated with the event of the particularevent type at the second content-addressable storage server, propagatingan event notification through the content-addressable storage cloud.Upon receipt of the event notification at the first content-addressablestorage server, the first content-addressable storage server may providefor notification to the first application of the event of the particularevent type. In some embodiments, notifications may be sent through thecloud using broadcasts or other methods. In various embodiments, eventsnotifications may be for storage, alteration, deletion, and/or readingof data in the content-addressable storage cloud or any otherappropriate event or action. Numerous other embodiments are describedherein.

These and other features and advantages of embodiments will becomeapparent from the following description of embodiments. Neither thissummary nor the following detailed description purports to define theinvention.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features will now be described with reference to thedrawings summarized below. These drawings and the associated descriptionare provided to illustrate specific embodiments, and not to limit thescope of the invention.

FIG. 1 illustrates a block diagram of an exemplary embodiment of asystem for management of interconnected content-addressable storagesystems.

FIG. 2 is a block diagram representing an exemplary process for eventnotification in a system of interconnected content-addressable storagesystems.

FIG. 3 is a block diagram representing a second exemplary process forevent notification in a system of interconnected content-addressablestorage systems.

FIG. 4 is a block diagram representing an exemplary process forproviding a shared archive in a system of interconnectedcontent-addressable storage systems.

FIG. 5 illustrates a second block diagram of an exemplary embodiment ofa system for content sharing with interconnected content-addressablestorage systems.

FIG. 6 is a block diagram representing an exemplary process for contentsharing in a system of interconnected content-addressable storagesystems.

FIG. 7 is a block diagram representing an exemplary process for creatingand managing virtual packages.

FIG. 8 is a block diagram representing an exemplary data structure anddata in a system of interconnected content-addressable storage systems.

FIG. 9 is a block diagram representing an exemplary process for virtualpackage management in a system of interconnected content-addressablestorage systems.

FIG. 10 illustrates abstract representations of virtual packages.

FIG. 11 is a block diagram representing an exemplary process for accesscontrol in a system of interconnected content-addressable storagesystems.

FIG. 12 is a block diagram representing an exemplary process forproviding a file system interface in a system of interconnectedcontent-addressable storage systems.

FIG. 13 is a block diagram representing an exemplary process forencryption in a system of interconnected content-addressable storagesystems.

DETAILED DESCRIPTION Overview

In the following detailed description, references are made to theaccompanying drawings that illustrate specific embodiments in whichembodiments may be practiced. Electrical, mechanical, programmatic, andstructural changes may be made to the embodiments without departing fromthe spirit and scope of the disclosure. The following detaileddescription is, therefore, not to be taken in a limiting sense and thescope of the disclosure is defined by the appended claims and theirequivalents.

Various embodiments overcome one or more issues with the prior art.Other embodiments may overcome different issues with the prior art. Forexample, some embodiments overcome issues related to event notification,others: virtual packages, yet others: shared archives. Some embodimentsovercome more than one issue.

Overview: Event Notification

Some of the embodiments allow applications to subscribe to “events” thatcorrespond to actions in the CAS cloud. The actions that may take placein the CAS cloud include, but are not limited to, the deposition,reading, alteration, or deletion of data. An application (or user) maysubscribe for all such actions, or for only those related to aparticular group, application, or individual performing the relatedaction. DICOM provides no such event notification. Providing this eventnotification can simplify the process for an entity to become aware, forexample, of new file about which it is interested, regardless of wherethey are deposited in the CAS cloud. In systems that did not takeadvantage of event notifications embodiments herein, learning of newfiles in which the application is interested can be complicated. If aradiology department in Virginia wanted to know if there were new X-raysto analyze from hospitals around the country, then they (e.g., the ITdepartment in Virginia) might set up polling programs that would checkspecific directories at each hospital (and possibly more than onedirectory at each hospital). Even in a static national networkenvironment without technical issues, this would be a cumbersometask—and would likely have many technical challenges related tocommunicating through firewalls, mixed authentication standards,varieties of network and communications capabilities, and ensuringcompliance to various standards (e.g., HIPAA or “Health InsurancePortability and Accountability Act”). In a more realistic networkenvironment, all of those problems would exist, and would be exacerbatedby the ever-changing network topologies, upgrading of software, storage,and computers, reconfiguration of computers, applications, and the like.In addition, it may be difficult or impossible for the hospital inVirginia to obtain access control information from the various hospitalsacross the country from which it is polling data. As an example ofsomething that would almost certainly happen without the hospital inVirginia's knowledge: a software administrator at one of the targethospitals changing the directory to which a PACS machine was storingX-rays (possibly to provide more storage). As a result, the pollingsoftware in Virginia might either stop working (e.g., if the previousdirectory disappeared) or not have visibility into updates (e.g.,because the new X-rays were being stored elsewhere).

These issues are overcome by some of the embodiments of eventnotification in a CAS cloud discussed herein. For example, a radiologydepartment's application may subscribe to the deposition of medicalimages by PACS machines into a particular file directory (perhapsmirrored to the CAS cloud as a bitfile)—or deposited directly into theCAS cloud. Those events may signal that the medical images should beanalyzed by the team in Virginia. The team in Virginia receives thosenotifications without needing to perform the technical connection to thecomputer or server that deposited the files—and without checking anydirectories. Further, the hospital in Virginia would receive eventnotification based on any events in that group (or any group to whichsubscribes). Further, as discussed with respect to embodiments herein,the CAS cloud can even provide the necessary access control. Relatedexamples are discussed more below with respect to FIG. 4.

Overview: Shared Archive and Related Data

Systems that do not provide a shared archive with searching, andautomatic related data searching, are inefficient and may be difficultand time consuming to use. Consider a network of DICOM servers that areall running independent of each other and store their files separately.Since DICOM does not provide a shared archive among DICOM servers, whena doctor wanted to find all of the medical images and reports related toa particular patent, particular examination, etc., the doctor would loginto a DICOM server and search for files. Certain files would bereturned, but the doctor would have no way of knowing if there wereadditional files to be found on other DICOM servers, some of which thedoctor may not even know about. Additionally, the separate DICOM (orother types of) servers may each have their own identifiers for patientsthat the doctor may not be aware of. Each server may also have separateaccess control (e.g., user name and password)—and the doctor may nothave log-in or other access information for each server. It would bedifficult, extremely time consuming, and in many instances, impossiblefor the doctor to obtain patient data from all of these servers.

Some embodiments herein provide parsing and archiving of data stored inthe CAS cloud, and later search and retrieval of that data, theunderlying files, and related data. The data may be stored to, forexample, a network-attached storage (NAS) or other computer hard driveby a PACS machine, and simultaneously or subsequently copied to the CAScloud. PACS machines may write data to drives as single files, oftencalled “blobs.” The CAS cloud, regardless of make or model of the PACSmachine, will parse the information newly-written to the CAS cloud andadd the information for the written data to the archive. For example,consider a PACS machine that writes a blob to its NAS drive thatcontains two X-rays and an MRI image with corresponding DICOM headers.An application connected to the CAS cloud, upon copying the file fromthe NAS drive, will parse the DICOM headers and store reference to thatfile and metadata (from the DICOM headers) in an archive. Embodiments ofmetadata management for metadata management are given in U.S. patentapplication Ser. No. 12/605,036, which is hereby incorporated byreference in its entirety for all purposes.

Often image and other medical files have metadata associated therewith,which may be termed “related metadata.” The term “metadata” as usedherein is a broad term encompassing its plain and ordinary meaning,including but not limited to “data or a set of data that providesinformation about the data with which it is associated.” For example, anX-ray or ultrasound file may have metadata attached thereto (e.g., as aDICOM header or XDS-I header) that describes the X-ray. The metadata maybe stored in a structured header, as plain text, etc. Further, somefiles may contain metadata. For example, an analysis or report maycontain metadata as part of the analysis or report (e.g., patient name,data, patient identifier, etc.).

A user with appropriate permissions could search the archive and find(and retrieve) the X-rays and/or the MRI. The user could also search forand retrieve data that had been stored by other PACS machines,ultrasound machines, etc. Additionally, the shared archive can look forrelated data. For example, if the doctor input a patient identifier forthe search, the shared archive may look up that patient identifier in amaster patient index to find equivalent entries. The shared archivecould then look up related data based on the other patient identifiersfrom the master patient index. Further, the shared archive may look uprelated data that is obtained based on data that is retrieved as part ofthe search. For example, if a query was for ‘all data on a patient’ (andassuming proper privileges for all data), the first response may be fora particular study. The study will have a unique identifier. The sharedarchive could then automatically query for all data related to thatstudy number. One of the results of for the study number query mayindicate that the patient previously had a different patient identifier.The shared archive could then look up all data for that previousidentifier. This process may continue until all appropriate related datais found. In this way, a shared archive with related data search notonly collects all of the related data in a way that it may behomogeneously searched, but also provides a way to find related data forqueries.

As used herein, the term “patient identifier” is a broad term,encompassing its plain and ordinary meaning, including, but not limitedto, “a number, reference, or code that represents a patient in a datasystem, such as a database, PACS, DICOM message, etc.” For example,patient identifiers may include name, social security number,system-specific patient number, Unique Patient Identifier, patientaccount number, etc. Patient identifiers may also be a combination ofone or more of these. For example, a patient identifier may be acombination of patient name and social security number. Patientidentifiers may also be unique to a patient across an organization,enterprise, nation, or all system (universal or global).

Overview: Virtual Packages

Grouping data can be difficult in medical systems, yet it is veryimportant. If a doctor would like to group all of the data related to aparticular patient, study, series, etc., that doctor would have to findall of the files and burn them to a CD, DVD, or, more likely, numerousCDs or DVDs. In current systems, all of this data may be stored onseparate PACS systems, DICOM servers, etc. As noted above and elsewhereherein, merely finding the data that the doctor would like to storepackage together is difficult or impossible in most systems. Finding thedata may require translating patient identifiers from those used in onesystem to those used in another, mapping accession numbers, manipulatingthe use of certain data records, etc. Further, these difficulties may beencountered for each server (e.g., a DICOM server or PACS server). Forexample, a doctor may not even know on which PACS, DICOM, or otherserver data resides. Even if the doctor knew on which server dataresides, the doctor may not be able to find data related to a particularpatient, study, series, etc. Further, even if the doctor could find allof this data, she may not be able to store the data to a single CD orDVD. Further, CDs and DVDs can get destroyed, lost, or, worse, fall intothe wrong hands. Additionally, the person distributing the package offiles may not be able to control the access to the files in the virtualpackage over time. For example, once a CD or DVD is distributed, even ifit has embedded access control, that embedded access control would bestatic and could not change over time.

Embodiments of virtual packages herein overcome one or more theseissues. In creating a virtual package, a doctor may chose to includerelated data, and, as described elsewhere herein, related data may beautomatically discovered and retrieved in a manner that, in many cases,might not be possible for the doctor to do alone. Further, after thevirtual package is made and stored in the virtual archive, the doctorcan distribute the virtual package without concern for the size of theunderlying files. Instead, the virtual package provides a smaller andmuch more manageable set of data than the underlying files.Additionally, the doctor need not be as concerned with ensuring properaccess control. The CAS system provides access control for the filestherein, including those in the virtual package. Further, the CAS systemcan provide access control that varies over time. As discussed in moredetail below, access control is being implemented by the CAS system atthe time that the files are being accessed. As such, if the accesscontrol needs to change over time (e.g., if a malpractice suit wasbrought, access to data related to that suite may be restricted,monitored, etc.), the CAS can provide that changing access control.

Further Overview

Some embodiments herein cover new approaches for the creation of andaccess to virtual packages in an interconnected content-addressablestorage system. Other embodiments herein provide new and noveltechniques for controlling access to data in a content-addressablestorage system. Additionally, various embodiments herein provideinterfaces for different filesystems heretofore unavailable in thecontext of content-addressable storage. For example, some of theembodiments herein provide a Common Internet File System (“CIFS”)interface. In addition, in some embodiments, data is protected (e.g., byencryption) as it is moved around the cloud, replicated, stored, and/orsent to applications using the cloud.

Details regarding several illustrative preferred embodiments forimplementing the system and method described herein are described belowwith reference to the figures. At times, features of certain embodimentsare described below in accordance with that which will be understood orappreciated by a person of ordinary skill in the art to which the systemand method described herein pertain. For conciseness and readability,such a “person of ordinary skill in the art” is often referred to as a“skilled artisan.”

It will be apparent to a skilled artisan, in light of this disclosure,that the system and method described herein can advantageously beimplemented using computer software, hardware, firmware, or anycombination of software, hardware, and firmware. In one embodiment, thesystem is implemented as a number of software modules that comprisecomputer executable code for performing the functions described herein.In one embodiment, the computer-executable code is executed on one ormore general purpose computers. However, a skilled artisan willappreciate, in light of this disclosure, that any module that can beimplemented using software to be executed on a general purpose computercan also be implemented using a different combination of hardware,software, or firmware. For example, such a module can be implementedcompletely in hardware using a combination of integrated circuits.Alternatively or additionally, such a module can be implementedcompletely or partially using specialized computers designed to performthe particular functions described herein rather than by general purposecomputers.

It will also be apparent to a skilled artisan, in light of thisdisclosure, that the modules described herein can be combined ordivided. For example, a skilled artisan will appreciate, in light ofthis disclosure, that any two or more modules can be combined into onemodule. Thus, referring to FIG. 1, the CAS server 121 and device managermodule 141 may be combined into a single module that performs thefunctions of both modules. Conversely, any one module can be dividedinto multiple modules. For example, the CAS server 121 can be dividedinto multiple modules such that each individual module performs part ofthe functions of the CAS server 121 and all of the modules collectivelyperform all such functions.

Similarly, a number of databases are described herein. A skilled artisanwill appreciate, in light of this disclosure, that any two or moredatabases can be combined into one database and that any one databasecan be divided into multiple databases. A skilled artisan will alsoappreciate, in light of this disclosure, that multiple distributedcomputing devices can be substituted for any one computing deviceillustrated herein. In such distributed embodiments, the functions ofthe one computing device are distributed such that some functions areperformed on each of the distributed computing devices.

The foregoing and other variations understood by a skilled artisan canbe made to the embodiments described herein without departing from thespirit of that which is disclosed herein. With the understandingtherefore, that the described embodiments are illustrative and that theinvention is not limited to the described embodiments, certainembodiments are described below with reference to the drawings.

Exemplary System

“Content-addressed storage,” or CAS, as used herein is a broad term,encompassing its plain and ordinary meaning, and includes techniques bywhich a unit of data stored on a storage system is accessed using anaddress that is derived from the content of the unit of data. The term“storage identifier” is a broad term, encompassing its plain andordinary meaning, including, but not limited to a number, reference,pointer, or object that references a memory location, location in a CASor CAS cloud, or other storage location. One type of storage identifieris a “GUID” or “globally unique identifier.” A GUID may be a globallyunique identifier that is made sequentially, based on a timestamp, etc.In some embodiments, notwithstanding the potential for collisions, aGUID may be created based on a hash (MD5, SHA, etc.). As an example, theunit of data may be provided as an input to a hashing function whichgenerates a GUID that is used as the content address for the unit ofdata. When a host computer sends a request to a content-addressablestorage server to retrieve a unit of data, the host provides the contentaddress of the unit of data (e.g., a GUID). The storage server thendetermines, based on the content address, the physical location of theunit of data in the storage server, retrieves the unit of data from thatlocation, and returns the unit of data to the host computer. As usedherein, the term “cloud” is a broad term, encompassing its plain andordinary meaning, including, but not limited to “a shared pool ofconfigurable computing resources (e.g., networks, servers, storage,applications, and services) that can be rapidly provisioned and releasedwith minimal management effort or service provider interaction.”

As depicted in FIG. 1, which shows a system 100 for content-addressablestorage, a cloud 110 of content-addressable storage servers 121-125, maycomprise numerous content-addressable storage servers, and thesecontent-addressable storage servers may be connected in any number ofways. In some embodiments, each content-addressable storage server maybe connected to every other content-addressable storage server (notdepicted in FIG. 1). In other embodiments, each content-addressablestorage server may be connected to one or more of the othercontent-addressable storage servers in the cloud. Therefore, anycommunication between two content-addressable storage servers may beover a direct connection, over a single path via one or more of theother content-addressable storage service, or over one or more ofmultiple paths via one or more of the other content-addressable storageservers. Various direct connections, single path connections, andmultipath connections, are depicted in FIG. 1.

The system 100 may include one or more CAS servers 121-125. Each ofthese CAS servers may include or have thereto attached one or morestorage servers (not pictured). The CAS servers may include one or moreRAID storage systems, cloud storage, tape storage, optical disks,magnetic disks, and/or any other appropriate type of storage. There maybe any number of CAS servers and each may have any number of storagesystems. Two or more storage systems may reside on the same physicaldisk or storage, or each storage system may reside on one or more disksor other storage separate from all of the other storage systems. In someembodiments, content stored in a CAS server 121-125 may be retrievedbased on a GUID and/or based on a search, as discussed herein.

As noted herein, when a CAS server 121-125 receives content to store, itmay store the content and metadata to the storage system. The contentmay be stored in the same format in which it is received on an attachedmemory. In some embodiments, other storage devices or methods may beused, such as flat files, directories, databases, or the like. Themetadata may be stored in any fashion, including in a database, flatfile, directory of files, or the like. In some embodiments, the metadatamay be stored in XML or other structured file as plain text and thisplain text may be searchable. In some embodiments, the metadata isstored in a database, and this database may be searchable.

Also depicted in FIG. 1 are three application modules 131-133. Theapplication modules 131-133 can be any type of applications that needstorage of data, particularly long-term storage of data that is notgoing to change quickly. The application modules 131-133, as discussedabove, may be medical application modules, and these medical applicationmodules may need access to medical images, such as X-rays, CT scans,MRIs, etc. that have been stored over a period of time. The applicationmodules 131-133 may communicate directly with the cloud 110. In someembodiments, as depicted in FIG. 1, application modules 131-133 maycommunicate with a device manager module 141-143, respectively. In someembodiments, device manager module 141-143 may provide an interfacebetween an application module 131-133 and the content-addressablestorage cloud 110, thereby providing application modules 131-133 with amore traditional data interface than they would have if they weredirectly coupled to the CAS cloud 110. For example, application modules131-133 may be able to request or search for data through the devicemanager module 141-143. Device manager modules 141-143, on the otherhand, may act on that request for that search by looking for a globalunique identifier, or GUID, that corresponds to the data (perhaps byquerying a GUID/Object database 151-153). Once the device managermodules 141-143 have the GUID, they will be able to request the dataassociated with the GUID either from an attached database (notpictured), or from the content-addressable storage cloud 110.

For example, if a PACS system 131 requests a particular X-ray fromdevice manager module 141, then the device manager module 141 may lookup the GUID for that X-ray in its GUID/object database 151. Once thedevice manager module 141 has the GUID corresponding to the X-ray, itmay request the object associated with the GUID from thecontent-addressable storage server 121. If the content-addressablestorage server 121 has the object associated with the GUID, then it willreturn it to the device manager module 141. If, however,content-addressable storage server 121 does not have the objectassociated with the GUID stored locally, then it may forward the requestfor the GUID to one or more other content-addressable storage servers inthe cloud 110. Each of the content-addressable storage servers to whichthe request for the file associated with the GUID was sent will check tosee if the object associated with the GUID is stored locally to it. Aswas the case for content-addressable storage server 121, thesecontent-addressable storage servers to which the request was forwardedwill return the object associated with the GUID to content-addressablestorage server 121, if it is stored locally. If the content-addressablestorage servers to which the request was forwarded do not have theobject associated with the GUID stored locally, then they will eachforward the request to one or more other content-addressable storageservers in the cloud 110. This process will continue until one of thecontent-addressable storage servers in the cloud 110 has the objectassociated with the GUID. Once the object associated with the GUID hasbeen located, it will be returned to content-addressable storage server121. Content-addressable storage server 121 will then return the objectto the device manager module 141, which will in turn provide the object,in this case a file containing an X-ray, to the PACS application module131.

In various embodiments, after the request for the object associated withthe GUID has been satisfied, the object associated with the GUID iscached or stored locally at the content-addressable storage server.Considering the example above, once the content-addressable storageserver 121 receives the object associated with the GUID, it may storethat object locally (e.g., at CAS server 121). As such any subsequentrequest for that GUID made to content-addressable storage server 121 mayproceed more quickly because the object associated with the GUID isalready stored locally at content-addressable storage server 121.

Given that storage is finite, a content-addressable storage server, suchas content-addressable storage server 121, may not be able to storeobjects associated with GUIDs indefinitely. Therefore, varioustechniques for offloading locally-stored objects to othercontent-addressable storage servers in the cloud 110 may be used. Forexample, in some embodiments, after a predetermined amount of time, forexample one day, one week, or one month, has passed, an object may be“timed out” and offloaded to another content-addressable storage systemin the cloud 110. This timeout technique may be used alone or inconjunction with other techniques. Another technique may be to monitorthe quantity or percentage of the storage capacity of thecontent-addressable storage server 121 that is currently in use. If acertain threshold amount of the storage capacity of thecontent-addressable storage server 121 is in use, then one or moreobjects may be offloaded to other content-addressable storage systems inthe cloud 110. The choice of which objects to offload may be based onany number of factors, such as the age of the object, the size of theobject, or some combination of the two. Many techniques are known tothose skilled in the art, including those that may predict which objectsare likely to be used in future. In some embodiments, the less likely anobject is to be used in the future, the more likely it should be anobject chosen to offload onto another content-addressable storage systemin the cloud 110.

A user, such as a doctor, may use the shared archive module 133 to queryfor particular studies (e.g., all X-rays for a particular patient takenin the previous 5 years). The shared archive module 133 may forward therequest to the data manager module 143. The data manager module 143 maylook up the GUIDs for the X-rays and request that data from the CAScloud 110. The CAS server 123 may then look locally for the dataassociated with the GUIDs. If the data associated with the GUIDs are notthere, then the request for the GUIDs may be sent into the CAS cloud 110to obtain the data associated with the GUIDs, as above. Once the CASserver 123 has the data associated with the requested GUIDs, it willsend that data to the data manager module 143, which will return it tothe requesting application, shared archive module 133. The user willthen have local access to the data in the CAS cloud 110.

FIG. 1 depicts a shared archive module 133 that has a shared archive CASmodule 143 and a database 153. FIG. 1 depicts the database 153 beingconnected to the shared archive CAS module 143, but the database 153 mayalso be directly connected to the shared archive module 133 or connectedto both the shared archive CAS module 143 and the shared archive module133. Further, in some embodiments, two or more of the shared archivemodule 133, shared archive CAS module 143, and database 153 may becombined as a single unit or module or may run on a single computer (notpictured).

The high-level overview illustrated in FIG. 1 partitions thefunctionality of the overall system into modules for ease ofexplanation. It is to be understood, however, that one or more modulesmay operate as a single unit. Conversely, a single module may compriseone or more subcomponents that are distributed throughout one or morelocations. Further, the communication between the modules may occur in avariety of ways, such as hardware implementations (e.g., over a network,serial interface, parallel interface, or internal bus), softwareimplementations (e.g., database, DDE, passing variables), or acombination of hardware and software.

Event Notification

Some embodiments allow users or applications to subscribe to particularevent notifications. As noted herein, DICOM does not provide eventnotification. Nevertheless, users of DICOM and similar protocols andsystems may desire or need event notification. Using embodiments herein,users or applications can subscribe to receive event notifications fromthe CAS cloud. The event notifications may correspond to or be triggeredby, events or actions taking place in the CAS cloud. For example,turning to FIG. 1, when a PACS application 131 stores an X-ray, MRI, orother medical image in the content-addressable storage cloud 110, viathe data manager module 141, that storage action may be associated withan event. Other applications, such as application 132 may subscribe toevents of a particular event type, such as the storage of X-ray imagesby the PACS application 131. When the PACS application 131 stores theX-ray image to the CAS cloud 110, an associated event notification ispropagated through the CAS cloud 110, as represented by the arrows inthe CAS cloud 110. For example, the PACS application 131 may store datain the cloud 110 via its local CAS server 121. Its local CAS server 121may forward a related event notification to the other CASs in the cloud110, such as CAS server 124. CAS server 124 may in turn forward theevent notification to CAS server 125 and, subsequently, CAS server 125may forward that event notification to CAS server 122. The devicemanager module 142 or the application 132 may then receive the eventnotification from CAS server 122. In order to receive the notification,the application 132 may communicate directly with CAS server 122 or todevice manager module 142 which may in turn speak to CAS server 122.Various other embodiments of event notification automation are describedherein.

As used herein, the term “event notification” is a broad term,encompassing its plain and ordinary meaning, including, but not limiteda notification that a particular event has taken place. The types ofnotifications and their delivery are discussed herein. The term “event”as used herein is a broad term, encompassing its plain and ordinarymeaning, including, but not limited to, execution or performance of aparticular action with respect to data in a CAS cloud or with respect tothe CAS cloud itself. Particular examples of events are describedherein.

FIG. 2 is a flow diagram depicting a method 200 for event notificationautomation. In block 210, a first system subscribes to an eventnotification for an event group “G.” Returning again to FIG. 1, thefirst system may be, for example, application 132. The group G mayrepresent any type of group for which application 132 may want toreceive event notifications. For example, the group G may correspond toall events for a particular department in a hospital, for a particularhospital, for a particular doctor for all medical images of a certaintype, such as X-rays, MRIs, etc.—or any other appropriate grouping. Thegroup G may be defined by any combination of one or more of patientidentity, doctor identity, hospital department, hospital, timing, or anyother appropriate aspect. The types of events in group G may also belimited to data being written to the CAS cloud 110, data being modifiedin the CAS cloud 110, and/or data being deleted from the CAS cloud 110.Additional grouping characteristics for event types for a group may alsoinclude, for example, storage, retrieval, migration, reloading,deleting, getting properties, changing groups, duplicating bit file,burning a volume, any check or analysis on the CAS cloud, anysynchronization or action such as a shutdown initialization, purging inthe CAS cloud, etc. In addition, application 132 may subscribe to eventsassociated with all actions in the CAS cloud 110. Turning to specificexamples of notifications, in some embodiments, all event notificationshave a Group ID (e.g., event group “G” discussed above) and GUID in thebody of the notification. An application or server may check allnewly-arrived event notifications and find those to which it issubscribed by matching or filtering for the Group IDs in which it isinterested.

Method 200 depicts only a single event notification subscription.Nonetheless, systems may subscribe to event notifications for additionalactions and/or for different groups other than group G. Variousembodiments will allow the first subscriber system to subscribe to eventnotifications for multiple groups and for multiple event types andperform the subsequent blocks 220-240 and optionally 250 for those otherevent notifications. For example, application 132 may subscribe toevents related to all deletions of X-ray files from a particularhospital, all additions of X-ray files from a particular hospital,and/or all maintenance issues with the CAS cloud.

After the first system subscribes to event notifications for group G inblock 210, a second system performs an event that triggers an eventnotification for group G in block 220. For example, if application 132subscribed to events for X-rays being written to the CAS cloud at aparticular hospital in block 210, and PACS application 131 triggers anevent in that group by writing an X-ray to the CAS cloud from thatparticular hospital in block 220. That event notification would getpropagated through the CAS cloud 110 to CAS server 122 as part of block230.

The forwarding of event notifications in the CAS cloud in block 230 cantake many forms in various embodiments. For example, turning to FIG. 1,each CAS server 121-125 may forward the event notifications to every CASserver 121-125 to which it is connected unless it is the CAS server121-125 from which it received the event notification. In this way, CASserver 121 would forward the event notification to CAS server 124 aswell as any other CAS servers to which it is attached. CAS server 124would then forward the event notification to all CAS servers 123, 125,and any others to which it is connected other than CAS server 121, fromwhich it received the event notification originally. Subsequently, eachreceiving CAS server will forward it to its other neighbors until allCAS servers in the cloud have received the event notification. In otherembodiments, the CAS servers may form a tree or hierarchy which definesto which neighboring CAS servers each CAS server should forward themessage, thereby eliminating some or all duplicate event notificationsbeing received at CAS servers in the CAS cloud 110.

In some embodiments, each event notification has associated with it anidentifier or signature. That identifier or signature may allow a CASserver 121-125 in cloud 110 to identify if the CAS server 121-125 hasreceived a duplicate event notification. Where appropriate, embodimentsignore duplicate event notifications and provide only the first ofduplicate event notifications to subscribers. The CAS servers 121-125may also forward only the first event notification received and, uponchecking the unique identified with subsequent event notifications, notforward any duplicate event notifications to its neighboring CAS serversin the cloud 110. An event notification may include a reference (e.g., aGUID) to the data associated with the action that was performed as wellas a group ID, timing, identity, and/or the type of event.

After the event notification is forwarded to the CAS cloud in block 230,the first system will receive the event notification for the event inblock 240. The first system may receive the event notification byperforming a polling action, such as running a recurrent script or aprogram that monitors events in the CAS cloud. For example, anapplication 132 may run a program that continually monitors a locationassociated with storing new event notifications in CAS server 122. Whena new notification arrives, the application 132 or the event monitoringprogram may compare the incoming notification's group ID with the groupIDs for which application 132 has subscriptions. If a matching group IDis found, then the application 132 is then aware of the new eventnotification. In some embodiments, the CAS server 122 and/or anotherapplication in the CAS cloud 110 may have a subscription manager (notdepicted in FIG. 1), which will push events to a device manager module142 or an application 132 in order to provide application 132 withupdates on events for which it has subscriptions. In such an embodiment,the subscription manager will check on newly arrived events in the CAScloud 110 and check the group IDs against the subscriptions ofapplication 132 and/or device manager module 142. If a matching group IDis found, the subscription manager will send the event to application132 and/or device manager module 142.

After the first system receives event notification for the event relatedto the action performed by the second system in block 240, the firstsystem may, optionally, retrieve data effected by or associated with theevent from the CAS cloud 110. For example, if application 132 hassubscribed to a group of events associated with the PACS application 131writing X-rays to the cloud 110, then upon the PACS application writingan X-ray to the cloud 110, application 132 should receive a notice(blocks 210-240). Application 132 may then retrieve the GUID from theevent notification and retrieve the data associated with the event fromthe CAS using the GUID. Application 132 can then retrieve the X-ray inblock 250. A subscription module may also automatically retrieve thedata associated with the event and send it to application 132 and/ordevice manager module 142. As noted above, retrieving data from a CAScloud 110 may involve querying the device manager module 142 or theapplication 132 may be able to request the file corresponding to theGUID directly from the CAS cloud 110.

Event notification embodiments herein may be implemented as a parallelto or as part of an implementation of the HL-7 standard. For example,event notification may be implemented to represent Instance AvailabilityNotifications to the systems using IHE/HL-7. The HL-7 InstanceAvailability Notification may be implemented, for instance, using theevent notification in the CAS as described herein. The storage of a filein the CAS cloud may trigger an event notification, sent as an InstanceAvailability Notification. The event notification may notify interestedworkflow actors (such as an IHE Department System Scheduler/OrderFiller, Post-Processing Manager and Report Manager, etc.) about theavailability status of data in the CAS. The messages may be sent usingan HL-7 Observation Results Unsolicited message, Medical DocumentManagement message, or any other appropriate message type.

Example Workflow Using Event Notification

FIG. 3 illustrates an abstract representation of an example workflow fortwo medical service providers connected to a content-addressable storageserver cloud. The first hospital is in California and the second isassociated with a radiology department in Virginia, both of which may beconnected to the same CAS cloud 110 in FIG. 1. The hospital inCalifornia subscribes to receive event Group “VA” event notifications inblock 310. The Group VA event notifications may be, for example, whenthe group in Virginia has stored reports on analysis of medical imageryin the content-addressable storage cloud. The radiology department inVirginia, on the other hand, may subscribe to receive event Group “CA”event notifications in block 311. The Group CA may correspond to X-raysbeing deposited or stored in the CAS cloud. After the two medicalservice providers have subscribed to events, they may later receiveevent notifications, as represented by the two dashed arrows in FIG. 3.In the example below, the California hospital performs an X-ray, theRadiology Department in Virginia analyzes the X-ray, and the Californiahospital receives a notification of that analysis. Many other workflowsare also possible with embodiments herein. Different imagery may be used(MM, ultrasound, etc.) or no imagery may be used at all. For example,the events and workflows may be related to analysis of written reports,proofreading transcription, or any other appropriate workflow oranalysis.

In block 320, an X-ray is performed at the hospital in California. TheX-ray may be performed using any known means, technique, or method. Inblock 330, the X-ray is stored in the content-addressable storage servercloud thereby triggering a Group CA event notification. Storing theX-ray to a CAS server or cloud may include writing it to a CAS server orcloud directly using a CAS interface, writing to a directory that ismirrored to a CAS server or cloud, writing to a file system interfacethat interfaces with the CAS server or cloud, or any other appropriatemethod, system, or technique. For example, referring to FIG. 1, a PACSapplication 131 or a device manager module 141 may write the X-ray to aNAS or other directory, which may then mirror the X-ray to the CASserver or cloud (possibly with header or other files or information aspart of the same file). “Mirroring” a folder or directory to the CAScloud 110 may take many forms, including replicating the folder as asingle bit file in the CAS cloud 110 or individual files in thedirectory may be stored in the CAS cloud 110 and the structure of thefolder (e.g., mapping of GUIDs for particular files to correspondingdirectories) may be stored separately (e.g., also in the CAS cloud 110or at PACS application 131 or device manager module 141).

Triggering the Group CA event may include content-addressable storagesystems that are interconnected in a CAS cloud 110 forwarding the eventsuntil each content-addressable storage system in the CAS cloud 110 hasreceived the event notification or at the very least until a CAS serverconnected to the radiology department in Virginia receives the eventnotification. Numerous examples of generating or triggering eventnotifications are described in detail elsewhere herein.

In block 340, the radiology department in Virginia receives the Group CAevent notification. Receiving that notification may spawn under numberof workflows including emailing a radiologist, populating a database orto-do list at a radiologist's workstation, inserting an X-ray into aradiology workflow or queue, etc. In this way a radiologist may be ablenotified that a new X-ray, is available and needs analysis.

In some embodiments, block 350 may include manually or automaticallyretrieving the X-ray from the CAS server or cloud after the eventnotification is received. Block 350 may also include the radiologistperforming analysis of the X-ray. After analyzing the received X-ray,the radiologist may store a report on that analysis into thecontent-addressable storage system or cloud, thereby triggering a GroupVA event notification in block 360. The report may include theradiologist's opinion, analysis, annotations, or any other informationor work product that has been made by the radiologist. As describedelsewhere herein, storing to a CAS server or cloud may include writingthe report to a directory and that directory being copied to thecontent-addressable storage server; writing to a network attachedstorage system, where that network attached storage system operating asan interface for the content-addressable storage system; storing thereport directly to a CAS server via a CAS interface or to the cloud; orany other appropriate writing means or technique. Referring to FIG. 1,either an application 132 or a device manager module 142 may control thewriting of the report to the CAS server 122 or CAS cloud 110.

The Group VA event notification generated by the radiologist's reportbeing stored on the content-addressable storage server or cloud (block360) is propagated through the content-addressable storage server cloudand received at the hospital in California in block 370. Receiving theGroup VA event notification in block 370 may spawn another workflow oractions, such as sending an email to an appropriate doctor, team ofdoctors, administrators, or other practitioners, etc. Receiving theGroup VA event notification in block 370 may be followed by storing anindication of the event in a database, file, or queue. Any otherappropriate notifications or actions may be taken based on the receiptof the Group VA event notification.

Once the event notification is received in block 370, the report oranalysis may be retrieved in block 380. The report may be retrieved inany appropriate manner, including those techniques discussed herein. Forexample, a doctor may retrieve the report from the CAS cloud 110 basedon the GUM in the Group VA event notification using an application131-133. The application may, in turn, use the data manager module141-143 to retrieve the report stored from the CAS cloud 110. Regardlessof which way the report or analysis is retrieved, the doctor may thenreview the analysis and continue patient care. From there the doctor maybe able to review that analysis and discuss it with a patient asappropriate. For example, using the techniques of FIG. 3, the doctor whooriginally performed the X-ray in California may later receive an emailthat the radiology department in Virginia analyzed the X-ray and can nowretrieve and review the report with the patient.

Numerous other workflows or methods may be performed using variousembodiments herein. Those embodiments may incorporate complex workflowsand notifications associated with each block in the workflow in order totrigger further action, such as depicted in method 300 of FIG. 3. Forexample, a technician may perform an ultrasound and store the ultrasounddata in the content-addressable storage server cloud. Storage of thatultrasound data may generate an event for a radiology department toanalyze it. The analysis may then be performed, and a report on theanalysis stored in the content-addressable storage server cloud, therebygenerating its own event notification. A doctor may then be notified ofthat the report on the ultrasound is ready for review. The doctor mayretrieve the ultrasound data and related report from thecontent-addressable storage server cloud and perform further review onthe ultrasound and the report. The doctor may then discuss the report orfindings with the patient and possibly perform another ultrasound. Thisfurther ultrasound may again be deposited in the CAS, which will againtrigger the radiology department to perform analysis. And the workflowmay be extended and/or repeated.

Shared Archive

Embodiments herein provide a shared archive in an interconnectedcontent-addressable storage system. For example, looking at FIG. 1, anapplication 132 and a PACS application 131 may each store information ina content-addressable storage server cloud 110. Upon storage ofinformation in the content-addressable storage server cloud 110, anapplication, such as shared archive module 133 or shared archive CASmodule 143, may parse the incoming files being stored in thecontent-addressable storage server cloud 110. Upon parsing theseincoming files, the shared archive may extract information from themetadata in those files and store the information (perhaps database153). As used herein and in this context, the term “information” is abroad term, encompassing its plain and ordinary meaning, including, butnot limited to “data or metadata in a parsed or otherwise accessibleform.” For example, when metadata is parsed, the information from themetadata may be accessible, for example, in RAM in a data structure, asentries in a database, etc.

At a later time, a user (not pictured in FIG. 1) may access the sharedarchive module 133 and query for files stored in the CAS cloud 110. Thequery may be for a particular type of image, such as an MRI image, aparticular patient, a particular doctor, or a particular hospital. Uponreceiving the query, the shared archive may search the database 153 forrecords that match the query and related data. The shared archive module133 may then return results to the requester (e.g., the one that sentthe query) in the form of GUIDs for the files that correspond to themetadata. The response to the query may be the metadata itself, a reporton the metadata, or any other appropriate information related to themetadata. In some embodiments, the shared archive may return all or apart of the file(s) matching the query.

Turning now to FIG. 4, a flow diagram depicts a method 400 for providinga shared archive. In block 410, a PACS machine stores a file containingmedical data (such as imaging data) and related metadata to a diskdrive, such as a network-attached storage drive. In this particularembodiment, the network-attached storage drive is mirrored to the CAScloud as part of block 410. This is represented with the mirroringinterface 471 depicted in FIG. 4. Mirroring a directory or drive mayinclude copying all files from one or more directories onto other drives(or into other directories). Redundant Array of Independent Disks (RAID)is one example of using mirroring techniques. If a local or networkeddrive (or directory) is mirrored to the CAS, then as files are copied tothat drive (or directory) are copied to the CAS. For example, turning toFIG. 1, if a PACS application 131 stores a file to a mirrored directory,the data manager module 141 may then copy that file from the mirroreddirectory to CAS server 121, which will return a GUID for the file. TheGUID may be stored in the GUID/OBJ database 151 along with informationabout the stored file. As used herein, the term “medical data” is abroad term, encompassing its plain and ordinary meaning, including butnot limited to images, videos, reports, and other medical-related dataassociated with one or more patients, accession numbers, study numbers,series numbers, etc. Medical data may include or have associated with itrelated metadata. Examples of medical data include, but are not limitedto DICOM images, DICOM reports, files in a proprietary format that haveheader data or metadata, XDS-I-encapsulated files or reports, or anyother appropriate data image or metadata.

Various embodiments provide for other sources of files for inclusion inthe shared archive. For example, block 411 depicts an applicationstoring a file, which contains a report and metadata, directly into theCAS cloud via a CAS interface 472. The CAS interface 472 is describedextensively above with respect to FIG. 1 and elsewhere herein. Anothersource of data is depicted in block 412. In block 412, an applicationwrites a file containing imaging data and metadata to a CAS cloud via afile system (e.g., CIFS) interface 473. A CIFS interface may beparticularly useful for legacy applications that communicate directly toa CIFS drive. That is, the application may be able to believe that it iswriting to a network-based CIFS drive and, in fact, the application isinstead writing to the CAS cloud. Implementations of network interfaces,such as CIFS interfaces, are described elsewhere herein and variousembodiments herein include other files system interfaces, such as NFS,FAT, SAMBA, etc. There may be additional ways to store files and datainto the content-addressable storage cloud and these are consideredwithin the scope of embodiments herein.

After a file is stored in the CAS, then, in block 420, the metadata ofthe stored file is parsed. In some embodiments, the stored files mayhave a single header containing metadata related to a report, image, orother documents stored therein. For example, an XDS-I file has a headerand an encapsulated file portion and the XDS-I header may have parsablemetadata which is parsed and used in the subsequent blocks of method400. As another example, a DICOM header may include a preamble and aprefix followed by numerous data elements. The header may contain datadescribing the data elements stored in the file. This header informationis parsed in order to extract information about the subsequent dataelements in the DICOM file. Some files stored in the CAS cloud are filesknown as “blobs.” Blobs may contain multiple DICOM files and/or otherformat files. For example, if a blob contains multiple DICOM files, theneach header may be separately parsed and information extracted about thedata elements following each header. A similar process occurs regardlessof the type of file stored in the CAS cloud.

Headers can be parsed based on the format of the file and header. Someheaders are in a mark-up language format and can be parsed based on thatmark-up language. For example, XDS-I headers are typically in XML or arelated mark-up language. Other headers, such as DICOM headers, may havefields that identify and define where and what type of information is inthe header. Other formats and methods of parsing them are alsoencompassed by the embodiments herein.

Once the metadata of the stored file is parsed in block 420, theinformation extracted from the parsed metadata is stored in the sharedarchive with a reference (e.g., a GUID) to the file itself. For example,if a DICOM image containing an X-ray is parsed in block 420 and patientname, doctor, time of performance of the X-ray, and hospital isextracted from the DICOM header in block 420, then in block 430 thisinformation is stored in the shared archive and associated with the GUIDof the stored DICOM file.

When information is stored in the shared archive in block 430 of FIG. 4,the shared archive module 133 may take the information obtained fromparsing the file stored in the CAS and store it in database 153 alongwith the GUID or other identifier that is associated with the metadata.In this way, a user may later be able to search the information in thedatabase 153 to find matching files and retrieve the GUIDs and/or thefiles.

Returning again to FIG. 4, there is a dotted line from block 430 back toblocks 410, 411 and 412. This dotted line represents that the actionsrelated to blocks 410-430 may repeat. Furthermore, the actions relatedto blocks 410-430 may repeat multiple times asynchronously with theactions in blocks 440 and 450. For example, multiple files may be storedin the CAS cloud sequentially or at the same time (blocks 410-412) andthe metadata for each of those files may be parsed (block 420) andstored in the shared archive (block 430). This process can continuenotwithstanding that queries may be received on the shared archive. Thatis, when a query is received on the shared archive (block 440), thequery will be run on whichever data has already been parsed from filesstored in the CAS (e.g., the latest information available).

In block 440, a query is received on the shared archive. The query maybe of any appropriate type (Boolean, SQL, natural language, DICOMQuery/Retrieve, etc.). The query may be for X-rays for a particularpatient, all images for a particular patient, all files (e.g., reports,images, etc.) for a particular patient, all files or a subset of filesfor a particular doctor, all files or a subset of files for a particulardepartment in a hospital or a particular hospital, etc. Additionally,the queries may be for particular periods of time, for example, allX-rays stored with relation to a particular patient between Jan. 1, 2009and Jan. 12, 2009.

Once the query is received in block 440, the query is run on the sharedarchive in block 450. Running the query may simply comprise running aquery on database 153. There may also be interpretation of the receivedquery in order to run the query on database 153. For example, the querymay request results related to a particular patient. Those results maythen be the basis of more queries for related data in block 451. Fromthe first results received, one or more queryable data items may beextracted. As used herein, the term “queryable data item” is a broadterm encompassing its plain and ordinary meaning, including, but notlimited to, a data item for which a search may be performed. In thiscontext, the queryable data item may be one or more data items, obtainedfrom the search results, for which subsequent queries or searches may berun. For example, a query for an X-ray could return an X-ray that isassociated with a particular accession number. The shared archive, inblock 451, may then perform another query, on the accession number, tofind related data. An “accession number” as used herein is a broad termthat encompasses its plain and ordinary meaning, including, but notlimited to, a number or identifier that identifies a particular objector collection of objects (such as a particular lab results or set of labresults).

Requesting related data can be a very complex undertaking. In aninterconnected system in which all of the interconnected systems andservers are using a shared identifier, requesting related data maysimply include sending requests to those connected servers and systemsfor all data related to that identifier. In many modern hospitals andhealthcare systems, however, different identifiers are used by differentsystems within the same interconnected medical environment. For example,one imaging server may identify patients by social security number.Another may identify patients by a patient identifier generated by thatsystem or some other centralized system. In addition, even if themultiple systems start with the same identifier, they may prepend orappend information such as time stamps or locations onto the identifierin order to identify that patient study or accession uniquely.

In addition to the difficulty related to the use of differentidentifiers, systems may codify or place identifiers in different orinconsistent locations. For example, consider a PACS system or otherimaging system in which there is only a single field for the patientidentifier. Some hospital systems may also use hierarchies of patientidentifiers in order to identify the patient correctly across a broadrange of systems. An imaging system that only has a single location tostore a patient identifier with a record may then have to be modified ortreated differently. For example, one of the two patient identifiersneeded may be stored in the space provided on that system for the singlepatient identifier. The other patient identifier on the other hand, maybe stored in another field, such as an accession field, middle namefield, or comment field. Further, because that other field is used for asecondary purpose, the information that would have originally gone intothat field may have to be placed in another location. This process maycontinue and numerous fields may be used inconsistently or in anonstandard way in the system. For example, turning to FIG. 8, there isa data structure 800 that contains a patient identifier field 810, astudy number field 820, and a hospital number field 830. As indicated bythe three dots, there may be numerous other fields. If a system needs tostore two patient indices, then data structure 800 may not providesufficient fields. The primary patient index 811 may have to be storedin the patient identifier field 810. The secondary patient index 812 maybe stored in the study number field 820, and the study number 821 may bestored in the hospital number field 830. Other changes may have to bepropagated through the rest of data structure 800. As a result ofinconsistent use of the field in the data structure, requesting relateddata from a system may require knowing how the data structures or otherpertinent identifiers are used within that system. A system wishing totalk to a PACS that uses the data structure 800 in FIG. 8 would have toknow to send its primary patient index in the data structures field 810,its secondary patient index in the study number field 820, the studynumber in the hospital number field 830, etc. Additional embodiments ofsearching for related data are discussed elsewhere herein.

As another example, a query for all data on a patient identifier (block450) may result in results associated with numerous accession numbers,study numbers, serried numbers, etc. In block 451, queries may be run onthe returned accession numbers, study numbers, series numbers, etc.Those queries may, in turn, results in other identifiers beingdiscovered (related to other accession numbers, series numbers, studynumbers, and possibly other patient or data indices). Each of those newidentifiers may also be searched and return results. In addition, thequery may be translated using a master patient index to run the query onother identities associated with the patient (e.g., other patientidentifiers, social security numbers, etc.). Master patient indexes aredescribed elsewhere herein. As an example, however, a patient may beassigned an index upon a first visit to the hospital, and, years later,be assigned another. This relationship may be stored in the masterpatient index. As another example, people may change their names overtime. The relationship between these two names may be stored in themaster patient index. For each of the queries for the identities foundin the master patient index, additional related data queries may also berun (block 451).

Blocks 450 and 451 may incorporate access control for queries. Theaccess control may be such that only particular users are allowed accessto particular types of data. As such, data and related data may not beretrieved in blocks 450 and 451 unless there is proper access rights, ordata may not retrieved and not sent to the requestor unless there areaccess rights (e.g., in block 470). Additionally, there may be classesor sets of users, for example, doctors may be allowed access to certaintypes of data, patients only to data about themselves, andadministrators may have access to all data. For example, block 440 mayinclude access control (460) and may not perform a query if therequester does not have proper access to data or information that wouldbe returned. Further (not depicted in FIG. 4), a user may not even beable to log into or present a query to a shared archive (which in someembodiments requires users to log in) if that user does not have properaccess. HIPAA provides requirements on data access control and dataencryption for some entities. In embodiments herein, these requirementsare implemented as part of block 450, 451, 460, or as part of otherblocks in FIG. 4. Turning to FIG. 1, the shared archive module 133and/or the shared archive CAS module 143 may provide access control. Theaccess control may be such that it corresponds to HIPAA's requirementsfor data security and integrity or any other appropriate access controlin which only certain users or classes of users have access to certaintypes of data.

Once the query is run and results are received, block 470 may includereturning those results to the requester (e.g., the person or systemthat sent the query). Returning the results to the requester may includereturning GUIDs to the data files stored in the CAS cloud that match thequery. The results may also be returned as a report or list of metadataor the files associated with the matching metadata may be returned. Forexample, if a requester requests X-ray files from Jan. 1, 2009 to Jan.12, 2009 for a particular patient, and three X-rays are found, then theshared archive module may return GUIDs associated with those threeX-rays or may return files containing the three X-rays, such as DICOMfiles (e.g., via a DICOM Store command).

As another example, an application may perform a DICOM Query/Retrieveusing embodiments of the shared archive. For example, the applicationmay perform a DICOM Query to the shared archive requesting data for apatient name, patient ID, accession number, physician, etc. The sharedarchive may receive the DICOM Query (block 440) and perform a search forthe patient name, patient ID, accession number, physician, etc. (block450). The shared archive may also query for related data (block 451).The shared archive may then return the results of the query (block 470).The requesting application may then perform a DICOM Retrieve for thestudies or other data that were returned as part of the DICOM Query.This DICOM Retrieve may be performed as part of block 440 or may beexecuted by a block not shown in FIG. 4. The shared archive may thenretrieve the data requested in the DICOM Retrieve and may return thedata to the requester as part of a DICOM Storage command. In someembodiments, the DICOM Retrieve may be sent to a server other than theshared archive. For example, the DICOM Retrieve could be sent to a CASserver and the CAS server could look up the GUID for the requested fileand retrieve and send the file (e.g., via a DICOM Storage message) tothe requesting application.

The discussion with respect to FIG. 4 and method 400 is primarilyfocused on files that contain medical data (e.g., reports or imagingdata) and metadata. Any other appropriate file may also be used. Forexample, a report stored in the CAS cloud may contain metadata and nothave a separate header. That report may be parsed and may be madesearchable in a shared archive using method 400. In addition, filescontaining test results may also be storable, parsable, and latersearchable. In this way, the shared archive module may providehomogeneous access to data that has been stored in a heterogeneousmanner in the CAS cloud 110. Such homogeneous access is particularlybeneficial considering that embodiments of the CAS cloud allow legacysystems to store to the CAS cloud (e.g., though mirroring or networkinterfaces to the CAS) in addition to newly developed systems to storeto the CAS cloud. In this way, various applications as well as multipletypes of data can be stored in the CAS cloud and accessed jointly andefficiently in a shared manner from the shared archive module.

Shared Archives and Document Sharing Protocols

Document sharing can be an important part of management andcommunication of information. In addition to the shared archiveembodiments described herein, various embodiments provide a documentsharing protocols (such as XDS and XDS-I) and, in some cases, integratedshared archives. There are many efforts to standardize the managementand sharing of documents. One such project is the organizationIntegrating the Healthcare Enterprise's (IHE) effort known ascross-enterprise Document Sharing (XDS). IHE has created a follow-onstandardization effort known as the cross-enterprise Document Sharingfor Imaging (XDS-I). XDS-I extends XDS by enabling the sharing,locating, and accessing of DICOM images and other documents that may bereceived from or accessed by, e.g., radiology centers, hospitals,doctors offices, or any other applicable source or consumer. XDS-I maybe useful for sharing X-rays, MRIs, and other health records, such ascardiology documents. XDS and XDS-I implementations may be built on theprinciple that one is able to query a single source in order to obtainaccess to stored documents from any of multiple repositories. As such,various embodiments herein may provide an XDS-I storage, searching, andretrieval interface that provides access to underlying documents storedin a CAS or CAS cloud.

As depicted in FIG. 5, various embodiments herein provide documentsharing by providing modules, such as XDS-I module 531. In someembodiments, XDS-I module 531 can address cross-enterprise EHRcommunication needs. Some scenarios may require the use of other IHEintegration profiles, such as Patient Identifier Cross-Referencing(PIX), Audit Trail and Node Authentication (ATNA), Enterprise UserAuthentication (EUA), Cross-enterprise User Authentication (XUA) andRetrieve Information for Display (RID), not depicted in FIG. 5.

As depicted in FIG. 5, cloud 510 includes multiple content-addressablestorage servers 520-522. A device manager module 541 may be incommunication with cloud 510. Device manager module 541 may haveassociated with it a GUID/object database 551 and may be coupled to anXDS-I module 531. Various embodiments of communication among clouds,device manager modules, GUID/object databases, and applications aredescribed with respect to FIG. 1. XDS-I module 531 may communicate withvarious XDS-I actors, such as an XDS-I source module 570 and an XDS-Iconsumer module 560. The XDS-I module 531 may also communicate withother XDS-I actors and/or other XDS-I modules, not pictured. In someembodiments, the XDS-I source module 570 may be coupled to an imagesource, such as a CT scanner or X-ray machine. In some embodiments, theXDS-I consumer module 560 may be coupled to a doctor's workstation or aPACS station, not pictured. Various embodiments of system 500 may allowa doctor to access the previously stored medical image, or any otherdocument, regardless of its source, using XDS-I.

Generally, in some embodiments, an XDS-I source module 570 may receiveimages from an imaging source, such as an MRI, X-ray, CT scan, or otherimaging device. The XDS-I source module 570 may then store the receivedimage in the cloud 510 by sending the image, using the XDS-I protocol,to the XDS-I module 531 (e.g., as a DICOM image encapsulated in an XDS-Imessage). The XDS-I module 531 may then send the image to the devicemanager module 541 which will then store it to the cloud 510 via thecontent-addressable storage server module 521, which will in turn returna GUID that corresponds to the stored image to the device manager module541. The device manager module 541 can store the relationship betweenthe stored image, its metadata, and the corresponding GUID in theGUID/object database 551. In some embodiments, XDS-I module 531 maystore metadata related to the stored image in addition to or instead ofdevice manager module 541. Subsequently, when an XDS-I consumer module560 sends a request to the XDS-I module 531 for that object, the XDS-Imodule 531 can request that object from the device manager module 541,which will then retrieve it from cloud 510, and send it to therequestor, possibly with additional related data. The described storage,searching, and retrieval of documents may apply, as noted above, todocuments other than images, such as electronic health records and thelike.

When the XDS-I source module 570 receives an image that it wants tostore, e.g., to cloud 510, it may encapsulate the image and add a headercontaining metadata about the image. The XDS-I source module 570 mayalso request any of a number of other transactions from the XDS-I module531. The transaction may include storing a document, altering a storeddocument, replacing a stored document, etc. Other example transactionsand embodiments of transactions are described in the appendices attachedto parent U.S. Provisional Application No. 61/327,556, which are herebyincorporated by reference for all purposes.

In some embodiments, when an XDS-I consumer module 560 searches for adocument or image, more than one document may match the request. Forexample, if an XDS-I consumer module 560 requests documents related to acertain patient, then the XDS-I module 531 may determine that multiplepatient records and images are related to that patient. The XDS-I module531 may then present to the XDS-I consumer module 560 a list of all ofthe matching documents. The XDS-I consumer module 560 may provide thelist of matching documents to a user, not pictured, that will allow theuser to select one or more of the matching documents for retrieval. TheXDS-I consumer module 560 may then request retrieval of all of theselected documents from device manager module 541, which may then lookup the GUIDs for those objects in its database 551 and request thoseobjects from the cloud 510. The XDS-I consumer module 560 may alsoperform other actions, such as (i) retrieve presentation states, whichrequests and retrieves the Grayscale Softcopy Presentation State (GSPS)information for a particular image or image set; (ii) retrieve reports,which may retrieve a diagnostic report, for example; (iii) retrieve keyimage notes; and (iv) retrieve evidence documents. More embodiments andexamples of various requests are described in the appendices attached toparent U.S. Provisional Application No. 61/327,556, which are herebyincorporated by reference for all purposes.

In some embodiments, the XDS-I module 531 may include or be attached toa metadata database or server, such as shared archive module 133, sharedarchive CAS module 143, or database 153. The XDS-I module 531 may storetherein metadata related to documents that it receives from XDS-Isources and relate them to the GUID that it receives from the devicemanager module 541 when the XDS-I module 531 stores documents to thecloud 510. For example, when XDS-I module 531 receives an XDS-I messageto store an X-ray from XDS-I source module 570, it may store metadatarelated to the X-ray (e.g., received as part of a header from the XDS-Isource module 570) in the metadata database along with the GUIDcorresponding to the stored X-ray image, received from device managermodule 541. In other embodiments, device manager module 541 may storethe metadata related to stored images in addition to or instead of XDS-Imodule 531 storing that data. Further, the device manager module 541 mayassociate the metadata for those images with the GUIDs for the image inthe database 551. In other embodiments, XDS-I module 531 may serve as ashared archive module and perform related functions, such as thosedescribed herein.

In some embodiments, when the XDS-I consumer module 560 requests adocument, the XDS-I module 531 may retrieve the requested document andother related documents and send them to XDS-I consumer module 560 in asingle response message. For example, if the XDS-I consumer module 560requests an X-ray for a patient, then the XDS-I module 531 may retrievethat X-ray as well as related medical data and return the X-ray and therelated medical data as a single XDS-I response. This may take the formof storing related medical data, such as lab results, patient name,etc., in an XDS-I header and the image in the XDS-I message.

In various embodiments, each module in system 500 may run on a separateprocessor or as part of a separate process. Two or more of the modulesmay be implemented together or may be run as part of the same process,on the same processor, on the same machine, as part of the sameapplication, and the like. For example, XDS-I module 531 and devicemanager module 541 may run as part of separate processes orapplications, or may be part of the same application or process. Asanother example, in some embodiments, XDS-I module 531 may also includeas part of the same process, or application, either or both of XDS-Isource module 570 and XDS-I consumer module 560.

FIG. 6 depicts a method 600 of providing XDS-I support. An XDS-I sourcemay receive a file to store from a user or machine, such as receiving anX-ray from an X-ray machine in block 610. For example, an XDS-I sourcemodule 570 may receive an X-ray image to store. In block 620, the XDS-Isource requests storage of the file in the XDS-I repository. Forexample, the XDS-I source module 570 may encapsulate the X-ray imagedata file in an XDS-I message and send the message to XDS-I module 531for storage (possibly using encryption for transmission and/or storage).In block 630, an XDS-I repository may receive a request to store adocument. For example, XDS-I module 531 may receive the request to storethe X-ray image data file encapsulated in the XDS-I message from XDS-Isource module 570. In some embodiments, an acknowledgment of storage maybe sent back to the XDS-I source, not pictured.

When an XDS-I consumer later requests documents, in block 640, thatrequest is sent to an XDS-I repository and matching documents, possiblyincluding related data and related documents, are determined as part ofblock 650. The XDS-I repository may then send a list of matchingdocuments to the XDS-I consumer, which will receive them in block 660.For example, XDS-I consumer module 560 may send the request fordocuments related to a particular patient to XDS-I module 531. XDS-Imodule 531 may then search for relevant documents and send the list ofthose documents to the XDS-I consumer module 560. Returning again toFIG. 5, the XDS-I consumer, in block 660, may request certain files thatwere found. In block 670, an XDS-I repository may send the requestedfiles to the XDS-I consumer, who may receive them in block 680. Forexample, XDS-I consumer module 560 may request certain files of thosethat were previously found in block 650 from the XDS-I module 531. TheXDS-I module 531 may retrieve those files from the cloud 510 via thedevice manager module 541 and send those files to the XDS-I consumermodule 560.

In some embodiments, a search requested in block 640 may include arequest to immediately retrieve all documents matching the search, andblock 650 may include retrieving those documents and sending them to theconsumer who will receive them in block 660.

In some embodiments, system 500 and method 600 may also include asecurity model that may be associated with an XDS-I Clinical AffinityDomain. For example, embodiments may group XDS-I Actors with actors fromthe IHE Audit Trail and Node Authentication (described elsewhere) andmay include access control capabilities that operate in cross-enterpriseenvironments. Example embodiments include Public Key Infrastructure, andthose related to ATNA, etc. In some embodiments, XDS-I modules may alsobe joined together or federated and thereby provide access to patientrecords as patients move from region to region, or country to country.

In addition to being able to provide functionality and support relatedto XDS-I, in some embodiments, a content-addressable storage servercloud 510 and/or interconnected set of content-addressable storageservers 520-522 may serve the function or provide storage for moregeneral XDS systems. Whereas XDS-I emphasized support for images, and inparticular medical images, the XDS protocol applies broadly to documentstorage, querying, and retrieval. Generally, an XDS Document Source mayprovide and register documents to an XDS Document Repository. Thedocument repository may register stored documents in an XDS DocumentRegistry. An XDS Document Consumer may query the XDS Document Registryin order to determine the locations of or references to documents. Oncethe XDS Document Consumer has received a response regarding locationfrom the XDS Document Registry, it may retrieve the documents from theXDS Document Repository. Queries from the XDS Document Consumer to theXDS Document Registry may include patient identification as part of thequery. An XDS Patient Identity Source may provide normalizinginformation for patient identity to be used in queries on the XDSDocument Registry. The XDS Patient Identity Source may provide anormalized identifier for a single person or patient across multiplehospitals, multiple systems, and the like. This is discussed more below.

Building on the discussion above of FIG. 5, in some embodiments, devicemanager module 541, since it may provide the functions of searching fordocuments and retrieving documents, may act as both an XDS DocumentRegistry and an XDS Document Repository. For example, in XDS DocumentConsumer may query device manager module 541, acting as an XDS DocumentRegistry, for the location of the document, not pictured in FIG. 5. Thedevice manager module 541 may then return a list of relevant documentsstored in the cloud 510 to the XDS Document Consumer. The XDS DocumentConsumer may then request some or all of the matching documents from thedevice manager module 541, which would then be acting as the XDSDocument Repository. The device manager module 541 could then retrievethe documents from the cloud 510.

In some embodiments, device manager module 541 may serve as an XDSDocument Registry in cloud 510 and/or a single content-addressablestorage server 520-522 or the cloud 510 may serve as the XDS documentrepository. If an XDS Document Consumer requested the location of an XDSdocument from the device manager module 541, acting as an XDS DocumentRegistry, may return the location of the requested document. The XDSDocument Consumer may then request the document from the cloud 510, oran individual content-addressable storage server 520-522, acting as anXDS Document Repository.

In any of these embodiments, an XDS Patient Identity Source may beimplemented as part of any appropriate module or system, such as devicemanager module 541 or a combination of multiple device manager modules541. In some embodiments, the XDS Patient Identity Source is implementedas part of cloud 510. Wherever it is implemented, in variousembodiments, an XDS Patient Identity Source may provide a master patientindex among numerous hospitals, doctors' offices, and the like. The XDSPatient Identity Source may include a database, data structure, or otherdata storage, with a unique identifier for each patient, and associationfrom that unique identifier to the identifiers from multiple independenthospital information systems and/or other systems.

For example, one hospital information system may identify patients bySocial Security number. Another hospital may identify patients by name.Yet other hospitals may identify patients by an identifier generated atthe hospital. The master patient index, running as part of devicemanager module 541, cloud 510, etc. in FIG. 1 (or as part of any otherapplication, device manager module, CAS server, CAS cloud, or as aseparate application), may store a single unique identifier for thepatient and associations from that single unique identifier to each ofthe identifiers from the various hospital information systems.Therefore, for example, a single patient John Smith may have a singleunique identifier in system 500. This single unique identifier may beassociated with his Social Security number at a first hospitalinformation system, his name for a second hospital information system,and the hospital—generated identifier for a third hospital informationsystem. Therefore, when a request is received for information related toJohn Smith, that request can be associated with his single uniqueidentifier and his single unique identifier can be used, in conjunctionwith its relationship with the other you identifiers, to query for andretrieve data related to John Smith from the multiple hospitalinformation systems, each of which may have data stored in the cloud 510(or elsewhere).

Virtual Packages

Traditionally PACS machines or other devices that manage medical imagesallow users to select files to write to a CD or DVD. Embodiments hereinprovide this functionality. In addition, certain embodiments herein willalso allow a user to produce a virtual package (or “virtual disk,”“virtual DVD,” “virtual file grouping,” etc.). The term “virtualpackage” as used herein is a broad term encompassing its plain andordinary meaning including, but not limited to, a grouping or set offiles, pointers to files, or other indicators of files. A virtualpackage may also refer to a single file including two or moresubsections containing images, reports, or other medical data. Variousembodiments herein allow a user to select a set of files to store in avirtual package. The system finds those files, and any related data, andsubsequently stores them in a CAS server or cloud. The system thenprovides the user with an indication of the virtual package or the filesin the virtual package (e.g., a GUID, set of GUIDs, etc.). Optionally,access control may also be included in the original request for thevirtual package. That is, the user may be able to specify what users orgroups of users are allowed to access all files within the virtualpackage (or perhaps a subset of files in the virtual package) and thecontent-addressable storage server or CAS cloud will then control accessbased on that information.

FIG. 7 illustrates an exemplary process for creating and managingvirtual packages. In block 710, an instruction is received to produce avirtual package. This instruction may be received at a computer orserver via a programmatic interface, for example, as an XML or otherstructured file received via a web service. The instruction to produce avirtual package may also be received via a web page or other interfacewhere a user can select files via drop down boxes, radio boxes,selection boxes, or other mechanisms. For example, a user at a PACSmachine running an embodiment would be able to select files to includein a virtual package (via a web-based selection menu, e.g.).

In some embodiments, a shared archive module or other program may allowa user to select files in the shared archive to include in a virtualpackage as part of block 710. Shared archives are discussed elsewhereherein. The files selected may be images, reports, or any other type ofdata. For example, a user may select all of the reports related to aparticular patient or a particular study, series, or SOP. The user mayalso select all of the files related to a particular accession number.The term “SOP,” as used herein is a broad term, encompassing its plainand ordinary meaning, including, but not limited to a “service objectpair” in DICOM.

In block 720, the files indicated by the user for inclusion in thevirtual package are obtained. In some cases, these files may be storedlocally to the PACS machine or other software that received theinstruction to produce the virtual package in block 710. In otherembodiments, a subset of the files may be stored locally. As such, arequest may be sent to a different computer or server to obtain one ormore of the files to be included in the virtual package. This requestmay be sent to a different PACS machine, a CAS server or cloud, adatabase, or any other appropriate storage location. Obtaining the filesmay include reading the files from a global storage system, such as CASserver or cloud. In yet other embodiments, obtaining these files mayinclude obtaining the files from the original request to produce avirtual package. For example, one or more of the files may be attachedto the request to produce the virtual package received in block 710.More specifically, if a user at an application were to select a numberof images and reports related to an accession number and attached a fewof those files to the request that is received in block 710, thenobtaining the one or more files may include retrieving those filesattached to the original request in addition to obtaining some of thefiles locally and/or from a global storage system, such as a CAS cloud.The files to include in the virtual package may also come from otherlocations as noted herein.

In various embodiments, the user may also request inclusion of relateddata the virtual package. The related data is requested in block 730.For example, if a user were to select files related to an accessionnumber, the user may also request that any data related to thataccession number be included in the virtual package. This may be useful,for example, in cases in which a user wants to include all availabledata related to an accession number, patient, study, or other grouping,in one virtual package so that subsequent users of the virtual packagemay have easy access to all of the related files. Requesting relateddata can be very complex, as discussed elsewhere herein.

As noted elsewhere herein, communicating to another system to findrelated data may also require translating an identifier using a masterpatient index. The master patient index may provide a single identifierfor each patient across multiple systems. Using the master patient indexto query for related data may include sending the master patient indexalong with the request for related data or it may include translating tothe destination system's patient index via the master patient index. Forexample, if a master patient index is being used and system A isrequesting from system B related data for a patient, then system A mayhave to convert its patient identifier into the master patient index andfrom there obtain the patient index used in system B. Similar structuresand techniques may be used for other unique identifiers or otheridentifiers such as accession number, study number, series number, andSOP number.

In some embodiments, requesting related data may include requesting datarelated to information that was received in the instruction in block710, such as accession number or patient number, or it may includeretrieving data from one or more of the files to be included in thepackage and using that data to request related data. For example, if apackage is being created for a patient that should include all of thefiles related to that patient, then the original request received inblock 710 may be used in order to identify the patient. Additionally,once files are obtained for that patient, accession numbers, studynumbers, series numbers, SOP numbers, and the like may be extracted fromone or more of those files and requests may be made for the data relatedto those accession numbers, series numbers, study numbers, and/or SOPnumbers.

In block 740, the requested related data is received. In addition, inblock 740 other related data may also be received even if it was notspecifically requested. For example, receiving the related data mayinclude receiving an HL7 broadcast message. The HL7 messages may includepatient related information and that patient related information may bematched to local patient information. For example, a PACS machine mayhave studies or X-rays related to a patient. An HL7 message may bebroadcast by another system that has related reports, other images,follow on images, etc. That incoming message may be matched by thepatient-related information, as discussed elsewhere herein. In someembodiments, the related data is included in the HL7 message itself. Inother embodiments, the HL7 message may not include the related data butmay indicate the source from which related data can be received. In suchembodiments, the related data may be requested from the machine orsystem that broadcast the HL7 message. After that data is requested, itmay be received by the requestor.

In some embodiments, the related data may be retrieved using DICOM queryand retrieve based on accession number as part of block 740 and/or block730. For example, if there is a DICOM server that has related data and asystem would like to request and receive that related data, the systemmay send a DICOM query/retrieve based on the accession number. In otherembodiments, the system may send the DICOM server a request for amodality worklist and match to that modality worklist and from there aquery based on the matches in the modality worklist. Receiving the datamay also include receiving and reading a DICOM Structured Report thatcame with various images. This DICOM Structured Report may be broadcastor otherwise sent to the receiving system with or without first queryingfor it. A DICOM Structured Report may be requested based on accessionnumber, patient number, study number, series number or SOP number. Inyet other embodiments, an HTTP request may be sent to a PACS server orother source and a web page, structured document, or the like may bereceived in response. This web page or structured document may be parsedto extract related data to be used in subsequent blocks of method 700.Related data may also be retrieved from a database using storedprocedures and/or SQL queries.

In some embodiments, when files and related data are received in block740, then that data is parsed and put in a canonical format so that itmay be stored in a coherent fashion in the virtual package. For example,receiving a DICOM Structured Report may be followed by parsing the DICOMStructured Report and storing the data in the database. Similarly, if aplain text file is received after a query for related data, then thatplain text file may be parsed and stored in a database. Subsequently,all of the related data stored in the database may be output in XML oranother format in order to store it as part of the virtual package. Invarious embodiments, the format to which the related data or obtainedfiles are converted may be DICOM or some other format such as a humanreadable format that is either standard, indicated in a configurationfile, or chosen by the user as part of the instructions to produce thevirtual package.

In block 750, the files and the related data are stored as a virtualpackage. A virtual package may take many forms. For example, the filesand related data obtained in earlier blocks may be stored individuallyin a content-addressable storage server. Upon storage of each of thesefiles, a GUID or other identifier may be returned. These GUIDs may thenbe used as identifiers inside the virtual package. A user of the virtualpackage may be able to retrieve the files in the virtual package basedon the GUIDs. In other embodiments, all of the files may be storedlocally grouped together as a single file (for example, as a tar ball orother grouping). This single file may be stored in the CAS server orcloud, at which time a single GUID may be returned. The single storedfile may also include metadata indicating the location identificationand other pertinent information related to each of the individual filesstored therein. In some embodiments, each individual file is stored inthe CAS server or cloud as described above and subsequently a merge isrequested in which all of the individual CAS files are merged togetheras a single CAS file and, optionally, the individual files are deletedfrom the CAS. In this way, the CAS server or cloud provides a GUID forthe combination of all of the files together. This procedure may alsoinclude creating and/or maintaining the metadata as described above forthe stored virtual package. Storing the individual files and/or thesingle file in the CAS may also be accompanied by keeping history data,auditing, and other information. For example, a file may be created or alog stored that indicates who created the virtual package, when theycreated it, its size, its contents, etc. This may be the case whethereach file is stored separately and those individual files are used asthe package, a single file is stored after being grouped together, orwhether the files are merged in the CAS.

In some embodiments, access control information is also provided to theCAS. The access control information may include email addresses, usernames, or other identifiers to indicate what individuals, what loginaccounts, what groups of people, or other groupings or individuals (forexample, such as systems) should have access to either individual filesin the virtual package or to the virtual package itself. As such, whenaccess is provided in block 760 to the virtual package the access tothat package may be controlled. For example, if access controlinformation is included for a virtual package, then a user who hasaccess to that package may receive an email indicating that the packagehas been stored in the CAS server or cloud and that that user may haveaccess to it at that point. In other embodiments, the user may have tosubscribe to an event notification in order to receive that email, asdescribed elsewhere herein. That user may be able to log into orotherwise authenticate to the CAS server or cloud in order to access thevirtual package. When attempting to access the virtual package, theaccess control of that user may be checked and only when that user hasappropriate permissions may the user access, modify, delete, or performother actions on either the individual files in the individual packageor the virtual package as a whole.

In some embodiments, other types of access control can also be providedby a CAS server, a CAS cloud, or other global storage system. Forexample, access to a particular patient, series, study, or accessionnumber may be turned off for all users. This may be useful, for example,if there is an issue with a patient, such as pending malpracticelitigation for that patient. In other embodiments, an individual user'saccess may be denied for all data in the global storage system. Forexample, if an administrator or other employee was fired then oneprecaution for the data in the CAS cloud may be to deny that person'saccount access to any data in the CAS regardless of previous permission.Controlling access in this way may be performed as part of a CAS server,CAS cloud, a separate application, or in any other appropriate manner.

Turning to FIGS. 1 and 7, an application 132 may receive a request toproduce a virtual package in block 710. Some of the files in the requestmay be attached to the request, other files may be stored locally inapplication 132, and additional files for the virtual package may bestored elsewhere. Application 132 may obtain those locally stored filesand other files either from the local disk, from the CAS cloud 110, orfrom the other sources (not pictured). The application may then send arequest for related data to the CAS cloud 110, a PACS application 131,and/or to a shared archive module 133. Each of those systems or devicesmay respond with related data in various formats. The application 132may then store the related data as a virtual package in the CAS cloud110, using the various techniques and embodiments described above. Theuser of the application may have also provided access controlinformation for the virtual package. Thereafter, the CAS cloud 110 maycontrol access to the virtual package requested and produced previouslyas part of block 760.

Various embodiments herein provide for the creation and accessing ofvirtual packages. When a user would like to create a collection of datathat might traditionally be put on the disk, the user may notnecessarily want to actually produce a physical disk. Therefore,embodiments herein allow the user to create a virtual package (perhapscalled a “virtual disk”). The virtual package may be more easilydistributed because it is not limited to physical disks. Further, asdiscussed herein, access can be finely controlled with a virtual packageand updated over time in ways that are difficult or impossible withphysical disks.

Additional Embodiments of Virtual Packages

This section provides more examples of virtual packages, their creation,and access. The contents of the virtual package could be, for example,from the CAS cloud 110. If a user is working at a PACS server 131, theuser may be able to create a virtual package. The virtual package mayinclude a number of objects, such as X-rays or MRIs, associated with aparticular patient or group of patients. The PACS server 131 maycommunicate with the device manager module 141 and request the GUIDs forthe objects in the virtual package. The device manager module 141 maythen look in the GUM/object database 151 in order to determine theobjects' associated GUIDs. The virtual package may simply be acollection of GUIDs.

When that same user or another user accesses the PACS server 131, orperhaps a different PACS server, in order to retrieve the virtualpackage, the PACS server 131 may communicate with the device managermodule 141 in order to retrieve all of the objects associated with theGUIDs that are associated with the virtual package. An abstractrepresentation of the virtual package is given in FIG. 10. In FIG. 10, avirtual package 1010 is associated with multiple GUIDs 1021-1029. When auser later attempts to access virtual package 1010 from the PACS server131, the PACS server 131 will access the stored representation of thevirtual package 1010 in order to obtain an indication of which GUIDs areassociated with virtual package 1010. The PACS server 131 will thenretrieve the objects associated with each GUID. Together, these objectsmake up the content of the virtual package. The user will then be ableto view, modify, or perform any other action on the virtual package,including, if appropriate, burning a physical disk based on the virtualpackage.

In some embodiments, virtual packages will include references to objectsinstead of GUIDs. Referring again to FIG. 10, a virtual package 1030maintains references to various included objects references 1041-1049.When a user later attempts to retrieve virtual packages 1030, theapplication module will ask for the various objects from the underlyingfilesystem, such as the device manager module 141 in FIG. 1. The devicemanager module 141 may then look in its own database 151 in order todetermine what GUIDs are associated with each of the object references.Once the device manager module 141 has the GUIDs for the objectreferences 1041-1049, it can request the objects associated with thedetermined GUIDs from the content-addressable storage cloud 110. Thecontent-addressable storage cloud 110 may then return the objectsassociated with the GUIDs to the device manager module 141. The devicemanager module 141 may then return the objects associated with theGUIDs, which are in turn associated with the object references1041-1049, to the requesting application module. The requestingapplication module would then have access to the contents of the virtualpackage.

Accessing a Virtual Package after Creation

FIG. 9 is a block diagram illustrating a method 900 for creating andaccessing virtual packages. Generally, in some embodiments, the virtualpackage may be created and associated with numerous files in thecontent-addressable storage system. After the disk is created, access tothat disk may be provided only to those who are allowed access. Further,physical disks may be burned based on virtual packages.

In block 910, a request for virtual package may be received. The requestfor the virtual package may come from a user using an applicationmodule, such as an application module 131 or 132, as depicted in FIG. 1.The request to create the virtual package may be received at the devicemanager module 141 or 142, as depicted in FIG. 1. In some embodiments, acontent-addressable storage server 121-125 in the cloud 110 may beresponsible for creating virtual packages. The request to create avirtual package may also include, depicted as block 920, a selection offiles to be placed on the virtual package. The selection of files may bestored locally at the receiver, such as device manager module 141 or142, or they may be stored on the content-addressable storage servers inthe cloud 110. The selection of files may also include related data, asdiscussed elsewhere herein.

In block 930, the selections of files are associated with virtualpackage. The virtual package may be associated with an identificationnumber, which may be stored in a database or other file, such asdatabase 151 or 152 in FIG. 1. Associating the files with the virtualpackage may include associating identifiers associated with each of thefiles to the identification number associated with the virtual package.As depicted in FIG. 10, a virtual package 1010 may be associated withidentifiers 1021-1029, and the identifiers for the files, when the filesare stored in the content-addressable storage cloud 110, may be GUIDs1021-1029.

At some point after the creation of one or more virtual packages, theapplication managing virtual packages may receive a request for aparticular virtual package, in block 940. The request to access aparticular virtual package may be a request that identifies the virtualpackage by an identification number. In some embodiments, such as thatdepicted in FIG. 9, when a request for virtual package is received, theapplication may determine in block 950 whether the requester is allowedto access the virtual package. If access is not allowed, then in block955, access is denied to the requester. If access is allowed, asdetermined in block 950, then the requester is given access to thevirtual package and may access files in the virtual package as part ofblock 960, burn a physical disk related to the virtual package in block965, or perform any other appropriate action.

Looking to the example of FIG. 1, if a device manager module 141 ismanaging virtual packages, then when someone using application module131 requests the virtual package, assuming that the requester is allowedaccess, the device manager module 141 may look up that virtual packageand its associated objects in the database 151. The virtual package maybe associated with a number of GUIDs, which represent the objects ordata files in the virtual package. When the requester attempts to accessthe files in the virtual package, the device manager module 141 mayrequest those files using the GUIDs from content-addressable storageserver 121. As described herein, if the content-addressable storageserver 121 does not have the data files stored locally, then it mayrequest those data files from other CAS servers in the cloud 110.

Access Control

In various embodiments, access control may also be provided by thecontent-addressable storage servers in the cloud 110. Access control mayfall into various categories, such as role-based access control anduser-based access control. Typically, role-based access control willrequire a user to login and that user will be associated with the role.The access that the user has to the data may be defined by a role.User-based access control is similar in that a user must login and thatuser will have access to the data based on his or her specific accessprofile. In various embodiments, the access control mechanisms maycontrol reading data, writing data, modifying data, deleting data, etc.Access control may be defined for individual objects, such as X-rays forparticular patients, or for types of objects or groups of objects. Forexample a given user may have access to all of the medical images for aparticular hospital and may not have access to medical images for anyother hospital (even if the other hospitals also have data stored in thecloud). On the other hand, a clinical research organization's analystmay have access to diagnostic results for a number of differenthospitals, but may not have access to any information that personallyidentifies any of the patient's with whom the results are associated.

In some embodiments, when either a user or a computer system attempts toaccess something in the content-addressable storage cloud 110,authentication information may be required. As used herein,authentication information may be a username and password, anidentifier, an RFID chip read by a device coupled to the system, or anyother authentication mechanism or technique known to those skilled inthe art. For example, a username and password may provide authenticationto access the cloud 110. If the username and password do not match orotherwise fail, then access to the cloud 110 may be denied.

In some embodiments, GUIDs associated with objects stored in the cloud110 may also be associated with a group number. Additionally, particularusernames and passwords may be associated with one or more groupnumbers. As such, when a particular user logs in using a username andpassword that particular user may have access to only the objectsassociated with the group numbers to which it has access. For example,consider a scenario in which multiple hospitals are all using the samecloud 110 to store their patient data. Each hospital may be associatedwith the group number. Each hospital may also have a username andpassword that users or computer systems associated with that hospitalmay use in order to access data. By providing this kind ofdifferentiated access control, the content-addressable storage systemmay effectively isolate the patient files for one hospital from those ofanother hospital. Such an arrangement may provide certain benefits. Byallowing multiple hospitals to store a patient data on the same cloud,each hospital may not necessarily have to build or support the physicalcomputing and communication infrastructure needed to support the cloud.The infrastructure may be provided by one of the hospitals or it may beprovided by a third-party. Either way, economies of scale may apply andthe storage of data on this cloud from multiple hospitals may providemonetary and other benefits.

FIG. 11 depicts a process for providing access control in a cloud ofinterconnected content-addressable storage servers. In block 1110, userauthentication information is received. The user authenticationinformation may be a username and password or other identifying orauthenticating information. In block 1120, a check is made to determinewhether the user authentication information is valid. This can includedetermining whether the username and password received from a user isvalid, or checking authentication with any other known method. If theuser is not authenticated, then access to the system is denied in block1130. If the user is authenticated, as determined in block 1120, thenthereafter the system may receive a request from the user for storeddata in block 1140. That request may come in the form of a request for aparticular GUID. Even though the user has already been authenticated,the user may not have access to all of the data stored in thecontent-addressable storage cloud, such as cloud 110. As such, in block1150, a check is made to determine whether the user has permission toaccess the requested data. Checking whether the user has access to therequested data may take a number of forms. For example, in role-basedaccess control, a determination may be made to see whether a roleassociated with the user, such as the role of analyst or administrator,provides that user with access to the requested data. Access control mayalso be finer grained. For example, access to files may be determined ona per user basis. As such, a check may be made to determine whether theuser is allowed to access the particular file. If the user does not havepermission to access the requested data, then access will be denied inblock 1160. If the user does have permission to access the data, then inblock 1170 access is provided.

Returning to FIG. 1, if a user is using a shared archive module 133 andattempts to access a file, such as an X-ray, that is stored in the cloud110, then the shared archive module 133 will request that file from theshared archive CAS module 143. The shared archive CAS module 143 willattempt to retrieve the file from the cloud 110. In some embodiments,the device manager module will check the access control information forthe user. In other embodiments, the shared archive CAS module 143 willprovide authentication information for the user to the cloud 110, and acontent-addressable storage server 123 will check to see whether theuser has proper authentication to access the requested file. If the userdoes have access to the requested file, then the cloud 110 will returnthe file to the shared archive CAS module 143, and the shared archiveCAS module 143 will send the file to the shared archive module 133.

Interfaces for an Interconnected Content-Addressable Storage System

In some embodiments, content-addressable storage servers in the cloud110 provide entities communicating with content-addressable storageservers interfaces that are similar to those normally associated withfile systems. For example, in some embodiments, a CIFS interface isprovided by the content-addressable storage servers in the cloud 110.Therefore, a program that is written to communicate with the familiarCIFS interface will be able to access the content-addressable storageservers. In other embodiments, other interfaces are provided, such asWFC interfaces. IHE Cross Enterprise Document Sharing (XDS) interfacesare provided in some embodiments. Additionally, some embodiments hereinmay be used as an XDS Document Repository, document registry, documentsource, or document consumer. One of the advantages of providing afamiliar interface is that it will allow a program written to work withthe familiar interface, even if the underlying storage is different, towork with little or no modification.

Returning again to the CIFS interface, the interface may act as a filteron requests for stored files. When a request for a file is sent to thefile system manager, the file system manager will check to see whetherthe file is stored locally in whole or in part, and if it is not, thenit will be requested via a CIFS call. The requested file will then beretrieved from a remote location, and sent to the requester.

As another example, consider the method depicted in FIG. 12. A filerequest is received in block 1210. A check is made to determine whetherthe file is stored locally, in whole or in part in block 1220. If thefile is stored locally, then in block 1230 the locally stored file isretrieved. Subsequently, the retrieved file is sent to the requester inblock 1250. If, however, it is determined in block 1220 that the file isnot stored locally, then the file is retrieved from thecontent-addressable storage server in block 1240. Examples of retrievingfiles from a content-addressable storage server are presented elsewhereherein. The retrieved file is then sent to the requester as part ofblock 1250.

There may be advantages to embodiments such as those depicted in FIG.12. For example, systems requesting files do not need to know whetherthe file is stored locally or stored remotely in the content-addressablestorage server. As such, in some embodiments, the management of thestored data is independent of the representation shown to the end-user.As such, the end-user, such as an application module 131, can operateindependently of the CAS mechanisms and independent of any changes tothe underlying file storage system. Therefore, in some embodiments, thecontent-addressable storage system as a whole may be modified or updatedwithout requiring any modifications or updates to an application module131 that may rely on the storage capabilities of the content-addressablestorage system.

Encryption for Transmission

The encryption protection provided in various embodiments may bemanyfold. Data may be encrypted as it is sent from onecontent-addressable storage server to another. It may also be encryptedwhile it is stored in the content-addressable storage server. Each ofthese provides a different kind of protection. Encryption duringtransmission of data will help protect the data from being “snooped” onthe transmission line or read by programs or individuals. Encryptionduring storage, on the other hand, will protect against the data beingreadily readable in the case that someone is attempting to read what isstored on the content-addressable storage server. Each of these twotypes of data protection is important for HIPAA.

Various embodiments herein provide for encrypted transmission betweenboth an application, such as an application module 131, and acontent-addressable CAS server, such as content-addressable CAS server121, as well as among various content-addressable CAS servers, such asthose in the cloud 110. Encryption works, generally, by takingunencrypted data and combining it with or modifying it based on acipher. The encrypted data can then only be read by decrypting the datausing a key corresponding to the cipher. Various embodiments ofencryption are known to those skilled in the art.

The system 100 of FIG. 1 may use encryption over one or more of thecommunication paths within the system 100. For example, data transferredbetween or among content-addressable storage servers in the cloud 110may be encrypted. The encryption may be transmission encryption, such asthat supported by secure socket layers (SSL) and secure hypertexttransfer protocol (HTTPS). Such encrypted transmission will allow atransmitting entity to send unencrypted data to another entity withoutseparately encrypting the data before the transmission. The encryptionis handled by the transmission protocol in these cases. On the otherhand, and possibly in combination with encrypted transmission, data canbe encrypted before it is sent. In such a case, the receiving entity mayhave the key or token associated with the cipher that was used toencrypt the data on the sender's side.

As another example, consider embodiments in which a data object, such asan X-ray, is being stored to the cloud 110 by an application module 131.The application module 131 may send the data object to the devicemanager module 141, possibly using encryption. The device manager modulemay then send, possibly using encryption, the object into thecontent-addressable storage cloud 110. In doing so, thecontent-addressable storage cloud 110 will return a GUID associated withthe object. The device manager module will then store the GUID in itsGUID/object database 151.

Considering the example system shown in FIG. 1, when the device managermodule 141 first send the object, possibly using encryption, in thecontent-addressable storage cloud 110 it will first go to a particularcontent-addressable storage server 121. Content-addressable storageserver 121 will at first store the object locally in its own storage.After some predetermined amount of time or once its data storagecapacity is reaching a certain predefined threshold, such as 50%, 80%,or 90%, certain amounts of data will be offloaded and sent to othercontent-addressable storage servers in the cloud 110, such ascontent-addressable storage server 123. In order to send the object fromone content-addressable storage server to another, certain embodimentsherein will first encrypt the object.

In some embodiments, encryption may be performed using data encryptionstandard (DES) or advanced encryption standard (AES) algorithms.Encryption may be performed by the content-addressable storage server121 or by any other appropriate computing device. Thecontent-addressable storage server 121 may also determine certain filetrailer or other metadata, such as a checksum or a digest, such as anMD5 digest. This file trailer or other metadata may be then sent to thesame content-addressable storage server to which the object is beingoffloaded. The receiving content-addressable storage server 123 may thenuse the file trailer or other metadata in order to determine whether thereceived object has been received correctly. If the received object hasbeen received correctly, then, in some embodiments, acknowledgments ofthis fact may be sent back to be sending content-addressable storageserver 121. If, however, the received object has not been receivedcorrectly then, in some embodiments, a request may be sent back to thesending content-addressable storage server 121 to resend the object. Theencryption of the objects when data is sent from one content-addressablestorage server 121 to another 123 may help provide safety and securityof patient data as it is in transit.

In some embodiments, objects and other data are encrypted when sentbetween any two entities, such as between the content-addressablestorage server 121 and the device manager module 141, and/or between adevice manager module 141 and an application module 131.

FIG. 13 depicts an example process for transmitting data usingencryption. In block 1310, a request is received to transfer a file. Thefile may be encrypted in block 1320, and the encrypted file may betransferred in block 1330. Encryption and transfer of data are describedelsewhere herein. The receiver may receive the file in block 1340 anddecrypt the file in block 1350. Decryption of data is describedelsewhere herein. As described above, a checksum or other data check mayalso be applied to the data to ensure the integrity of the receiveddata. In performing a process such as that depicted in FIG. 13, anunencrypted file may be received at a receiver, without that file everbeing in transit in an unencrypted state.

Additional Embodiments

The processes and systems described herein may be performed on orencompass various types of hardware, such as computer devices, such ascomputer systems. In some embodiments, application modules 131-133,XDS-I modules 531, 560, and 570, or the machines running applicationmodules 131-133, XDS-I modules 531, 560, and 570, the device managermodules 141-143 or the machines running device manager modules 141-143,the databases 151-153, 551 or the machines running databases 151-153,551, and the CAS servers, e.g. 121-125, 520-522, in clouds 110, 510 maybe each be separate devices, applications, or processes or may run aspart of the same devices, applications, or processes—or one of more maybe combined to run as part of one application or process—and/or each orone or more may be part of or run on a computer system. A computerdevice or system may include a bus or other communication mechanism forcommunicating information, and a processor coupled with the bus forprocessing information. The computer devices or systems may have a mainmemory, such as a random access memory or other dynamic storage device,coupled to the bus. The main memory may be used to store instructionsand temporary variables. The computing devices or systems may alsoinclude a read-only memory or other static storage device coupled to thebus for storing static information and instructions. The computersystems may also be coupled to a display, such as a CRT or LCD monitor.Input devices may also be coupled to the computer system. These inputdevices may include a mouse, a trackball, or cursor direction keys.

Each computing device may be implemented using one or more physicalcomputers, processors, embedded devices, field programmable gate arrays(FPGAs) or computer systems or a combination or portions thereof. Theinstructions executed by the computing device may also be read in from acomputer-readable medium. The computer-readable medium may benon-transitory, such as a CD, DVD, optical or magnetic disk, flashmemory, laserdisc, carrier wave, or any other medium that is readable bythe computing device. In some embodiments, hardwired circuitry may beused in place of or in combination with software instructions executedby the processor. Communication among modules, systems, devices, andelements may be over a direct or switched connections, and wired orwireless networks or connections, via directly connected wires, or anyother appropriate communication mechanism. Transmission of informationmay be performed on the hardware layer using any appropriate system,device, or protocol, including those related to or utilizing Firewire,PCI, PCI express, CardBus, USB, CAN, SCSI, IDA, RS232, RS422, RS485,802.11, etc. The communication among modules, systems, devices, andelements may include handshaking, notifications, coordination,encapsulation, encryption, headers, such as routing or error detectingheaders, or any other appropriate communication protocol or attribute.Communication may also messages related to HTTP, HTTPS, FTP, TCP, IP,ebMS OASIS/ebXML, DICOM, DICOS, secure sockets, VPN, encrypted orunencrypted pipes, MIME, SMTP, MIME Multipart/Related Content-type, SQL,HL7 Version 2.5, HL7 Version 2.3.1, etc.

As will be apparent, the features and attributes of the specificembodiments disclosed above may be combined in different ways to formadditional embodiments, all of which fall within the scope of thepresent disclosure.

Conditional language used herein, such as, among others, “can,” “could,”“might,” “may,” “e.g.,” and the like, unless specifically statedotherwise, or otherwise understood within the context as used, isgenerally intended to convey that certain embodiments include, whileother embodiments do not include, certain features, elements and/orstates. Thus, such conditional language is not generally intended toimply that features, elements and/or states are in any way required forone or more embodiments or that one or more embodiments necessarilyinclude logic for deciding, with or without author input or prompting,whether these features, elements and/or states are included or are to beperformed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art.

All of the methods and processes described above may be embodied in, andfully automated via, software code modules executed by one or moregeneral purpose computers or processors, such as those computer systemsdescribed above. The code modules may be stored in any type ofcomputer-readable medium or other computer storage device. Some or allof the methods may alternatively be embodied in specialized computerhardware.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

Following this document are a number of other documents, appendices,figures, diagrams, and publications. Each of these is incorporatedherein by this reference in its entirety and made a part of thisspecification.

What is claimed is:
 1. A method for event notification in aninterconnected content-addressable storage system, said method executingon one or more computing devices, said method comprising: receiving arequest for event notification of a particular type of event at a firstcontent-addressable storage server from a first application, said firstcontent-addressable storage server being implemented on one or morecomputing devices; receiving a request to perform an action associatedwith an event of the particular event type at a secondcontent-addressable storage server, wherein the first and secondcontent-addressable storage servers are distinct and wherein the firstand second content-addressable storage servers are both part of acontent-addressable storage cloud; performing the action associated withthe event of the particular event type at the second content-addressablestorage server; upon performance of the action associated with the eventof the particular event type at the second content-addressable storageserver, propagating an event notification through thecontent-addressable storage cloud; and upon receipt of the eventnotification at the first content-addressable storage server, providingfor notification to the first application of the event of the particularevent type.
 2. The method of claim 1, wherein the method furthercomprises, upon performance of the action associated with the event ofthe particular event type, propagating data related to the actionthrough the content-addressable storage cloud.
 3. The method of claim 1,wherein the method further comprises, upon notification to the firstapplication of the event of the particular event type, retrieving datarelated to the event notification at the first content-addressablestorage server.
 4. The method of claim 1, wherein providing fornotification to the first application of the event of the particularevent type comprises pushing a message related to the event of theparticular event type to the first application.
 5. The method of claim1, wherein providing for notification to the first application of theevent of the particular event type comprises the first applicationpolling for the event of the particular event type.
 6. The method ofclaim 1, wherein providing for notification to the first application ofthe event of the particular event type comprises the first applicationfiltering arriving event notification for the event of the particularevent type.
 7. The method of claim 1, wherein propagating an eventnotification through the content-addressable storage cloud comprisesbroadcasting the event notification to one or more neighboringcontent-addressable storage servers.
 8. The method of claim 1, whereinreceiving the request for event notification of the particular eventtype of event comprises receiving a request for event notification ofstorage of data.
 9. The method of claim 1, wherein receiving the requestfor event notification of the particular event type of event comprisesreceiving a request for event notification of alteration of data. 10.The method of claim 1, wherein receiving the request for eventnotification of the particular event type of event comprises receiving arequest for event notification of deletion of data.
 11. A system forevent notification in an interconnected content-addressable storagesystem, said system comprising: a first content-addressable storageserver comprising one or more computing devices configured to: receive arequest for event notification of a particular type of event; and uponreceipt of an event notification of the particular type of event,providing for notification to the first application of the event of theparticular event type, wherein the event notification was received froma second content-addressable storage server, wherein the secondcontent-addressable storage server: received a request to perform anaction associated with an event of the particular event type; performedthe action associated with the event of the particular event type; andupon performance of the action associated with the event of theparticular event type at the second content-addressable storage server,sent the event notification to the first content-addressable storageserver.
 12. The system of claim 11, wherein the firstcontent-addressable storage server is further configured to, uponnotification to the first application of the event of the particularevent type, retrieve data related to the event notification.
 13. Thesystem of claim 11, wherein providing for notification to the firstapplication of the event of the particular event type comprises pushinga message related to the event of the particular event type to the firstapplication.
 14. The system of claim 11, wherein providing fornotification to the first application of the event of the particularevent type comprises the first application polling for the event of theparticular event type.
 15. The system of claim 11, wherein the eventnotification received at the first content-addressable storage server isreceived from the second content-addressable storage server through acontent-addressable storage cloud that comprises the first and secondcontent-addressable storage servers.
 16. The system of claim 11, whereinreceiving the request for event notification of the particular eventtype of event comprises receiving a request for event notification ofstorage of data.
 17. A non-transient computer-readable medium comprisingcomputer-executable instructions for event notification in aninterconnected content-addressable storage system, saidcomputer-executable instructions, when running on one or more computers,performing a method comprising: receiving a request for eventnotification of a particular type of event at a firstcontent-addressable storage server from a first application, saidcontent-addressable storage server being implemented on one or morecomputing devices; receiving a request to perform an action associatedwith an event of the particular event type at a secondcontent-addressable storage server, wherein the first and secondcontent-addressable storage servers are distinct and wherein the firstand second content-addressable storage servers are both part of acontent-addressable storage cloud; performing the action associated withthe event of the particular event type at the second content-addressablestorage server; upon performance of the action associated with the eventof the particular event type at the second content-addressable storageserver, propagating an event notification through thecontent-addressable storage cloud; and upon receipt of the eventnotification at the first content-addressable storage server, providingfor notification to the first application of the event of the particularevent type.
 18. The non-transient computer-readable medium of claim 17,wherein said method further comprises, upon performance of the actionassociated with the event of the particular event type, propagating datarelated to the action through the content-addressable storage cloud. 19.The non-transient computer-readable medium of claim 17, wherein saidmethod further comprises, upon notification to the first application ofthe event of the particular event type, retrieving data related to theevent notification at the first content-addressable storage server. 20.The non-transient computer-readable medium of claim 17, whereinreceiving the request for event notification of the particular eventtype of event comprises receiving a request for event notification ofstorage of data.